Trat bus station to Hat lek (Thai Border) is the fastest and most comfortable option.
Price is 120 Baht. They leave every hour, and the trip takes about 1.5 hours.
Another option is to take a songtaew (2 rows taxi) from the taxi station behind the day market. Those taxis are cheaper and make irregular stops along the way. Both minivans and taxis stop for a while at Klong Yai, but while the minivans would only stop for a few minutes, the songtaew may take a longer break.
You can purchase your visa for Cambodia at the border for 1200 Baht and one passport photograph (note: US$ not accepted). You can purchase an e-visa over the internet prior to your trip (price is in US$) From the Thai/Cambodian border you can take a motorcycle taxi or a taxi to Koh Kong.