The best way to get rid of caterpillars is to spray Bt. Bt is short for Bacillus thuringiensis which is a bacterium that produces a protein that when ingested by caterpillars, it destroys the caterpillar’s stomach. This prevents them from eating and causes them to die. It only affects caterpillars making it very safe to use but also that means if it isn’t caterpillars chewing on your plants it won’t do anything. (There are a few other Bacillus species that target mosquitos, leaf beetles, etc.)
Bt can be found as an Organic Certified spray at most garden stores. We use it every year right around now. You usually will find it in a liquid form (occasionally it is in a powder).
To use Bt you need to follow mixing directions on the bottle and a sprayer. You can use a large pump sprayer if you have a lot to spray or a spray bottle if you only have a few plants. Once you have mixed your spray, spray the leaves directly and make sure to coat the tops and bottoms of the leaves. Bt must be ingested by the caterpillar so you want to make sure all of the leaves are coated. It is best practice to spray at least 3 days in a row and try to avoid spraying before it rains. If it does rain in between sprays just add another spray afterwards.