The Hudson River offers beautiful views and undoubtedly offers numerous places to paddle but the two places I put in were Coldspring (just south of the main town dock where the gazebo is (going through a boat yard) and in Beacon near the train station. From both these places it is possible to paddle to Bannermans Island and or Fishkill creek. Bannermans Island is interesting to look at (trespassing is not permitted and the river is patrolled) because it has the remains of an old castle that was built by Bannerman who was a munitions manufacturer and had kept his munitions stored there.
Fishkill Creek enters the Hudson just north of Bannermans Island and is a nice place to get out of the winds and explore. The mouth of it isn't very obvious and you have to paddle underneath the railroad tracks, a low bridge or underpass, which would keep out any large boats.
Wind is always a consideration on the Hudson and the couple of times I went it seemed that the wind increased as the evening came. There are places all along the shores where one can stop, where there are no houses. Fishkill creek is blocked by a damn about a mile into it.
North of Interstate 84 at Chelsea is an excellent place to put into the Hudson but it is a bit far to get to Bannermans Island or Fishkill creek but one could head north and get to Wappingers Falls creek which I haven't yet done.
The towns have restaurants and Coldspring is a very tourist oriented town with many antique shops.
Coldspring, Beacon and Chelsea are all located on the east side of the Hudson off of rt. 9D which follows the Hudson.