Yellow caterpillars are larvae that turn in to stunning moths or butterflies. Some types of yellow caterpillars are fuzzy-looking and others have smooth segmented bodies typical of many species of caterpillars. Sometimes, yellow fuzzy caterpillars can be poisonous. Although they are not toxic enough to kill you, touching them can cause skin irritation.