The East River is the border between Madison and Guilford. It empties into Long Island Sound and, of course, the river is tidal. You can also paddle the Neck River, which joins the East just upstream of the state launch.
The East and Neck Rivers are in the East River Marsh complex, which I found surprising clear of invasive phragmite reeds that have ruined many of our tidal estuaries. This makes for some excellent bird sightings - several osprey, egrets, more willets than I've ever seen in one place, and two glossy ibises.
There is a tidal current, but I made decent progress against it. I was out for over 3 hours, but it may be possible to paddle far enough upstream to make a good long day of it. You can also access Long Island Sound from this location.
The lower part of the river is in flat level treeless terrain - on a windy day you won't have ocean waves, but you will have ocean winds. I have not been in the river at low tide, but there is a marina that launches smaller powerboats (the RR bridge must be a tight fit for them) upstream of the I-95 bridge.
I suspect that the lower East River is deep enough for canoes and kayaks at any tide.
Here's a link to my blog report on this trip, with photos:
Two boat launches. The free (DEP) state launch is on the Madison side of the river past the end of Neck Road (it is signed when you get that far). It has parking for a couple dozen vehicles and a sani-can.
The Guilford town launch may or may not charge for hand-launched boats (some towns in CT wave the boat launch fee for hand launches).
For the free state launch, take Neck Road south off of the Boston Post Road. Neck Road is 1.4 miles west on the Boston Post Road of Madison town center. Just keep going until you run out of road.