I'm always torn when it comes to taking worms from the garden. They are, after all, responsible for much of the soil's fertility (the worm's excretions are seven times richer in plant foods than the organic debris they ingest). In addition, their vigorous tunneling provides channels for drainage and aeration, far deeper than a gardener will ever dig. Much better to find a field where they can be reliably harvested on damp nights. Stored carefully in a cool place and fed on vegetable scraps, these can last for several weeks.
The supply of redworms and brandlings from the manure pile, however, seems to be inexhaustible, so I don't mind taking those, although they're less useful, perhaps, because of their smaller size and more fragile nature. Roach and bream seem to find redworms almost irresistible at times and boy, do they wriggle!