OK here we go. I knew the chemistry degree would come in useful some day!
Polyvinyl alcohol is used as a thickener,
stabilizer, and binder in cosmetics, paper cloth, films, cements and mortars. Like most molecules with alcohol groups on them it is hydrophilic which means that to some degree it is soluble in water. What inhibits its solubility is the sheer size of the molecule. It it this property that makes it so useful to us anglers.
It is a fully saturated molecule (all single bonds for the techies)which means that it is pretty stable and not likely to be at all carcinogenic (can't find any literature on this so you'll have to take my word for it). It is biodegradeable in the environment although I'm not sure if there is any research on how this occurs in solution.
It is non-toxic and it is often used in school experiments to produce a slime similar to the stuff you can buy in toy shops. This is done by treating a solution of pva with borax if you want to try it at home. The borax removes the hydrogen from the alcohol group and allows cross-linking to occur (again only for techies!).
The reference Ron made to producing toxic fumes on burning doesn't make a lot of sense. I think this may be a reference to partial combustion which can produce nasties from any hydrocarbon compound, even sausages! I therefore wouldn't recommend you putting pva in your roll-ups, although it is like likely to be less toxic than tobacco when burnt!
To sum up. I would only be concerned about the effect of pva when it reaches high concentrations in water (much higher than we're likely to get from dissolved pvs bags). Again, there is no research that I can find, but I would suspect that in high concentrations it might affect dissolved oxygen content and the take-up of oxygen by the fishes gills (dissolved substances can have a pretty dramatic effect on processes such as osmosis). My gut feel is that this is unlikely to happen to any great extent as much of the dissolved pva will biodegrade before this happens.
Hope that helps!