Ok. You do not want to remove the chain. Fair enough. If the limit adjuster isn't set properly, NO amount of playing with the barrel adjuster will fix your issue. And, if it's a barrel issue, this will get you back to ground zero. Win/win!
1- Run the shifter all the way down until it won't "click" any more.
2- Loosen the barrel adjuster so that the cable is slack between where the cable housing is anchored to the derailleur body and the clamp for the cable.
3- Go back to the lever, and try to click it down some more. IMPORTANT.
4- Start rotating the cranks, and see if it still doesn't want to engage the smallest cog, we are almost done!. At this point, it will either not drop down at all, or start going "Gung...Gung...Gung" (in a metallic fashion
) Like it wants to drop down, but it just won't.
5- Find the limit screw on the derailleur that allows the body to go farther outboard (away from the bike). Start by rotating it about 1/2 turn, and rotate the cranks again. If it is still not there (as in going into the smallest cog), allow more movement outboard until it hits the stop. KEEP THE CABLE SLACK!
6- Once again, go back to the shifter, and click it all the way down. We are just making sure again.
7-IMPORTANT. Make sure that the derailleur cable is still slack during this entire process. If it get's tight while you are adjusting the limit screw, you are not at the mechanical limit (metal/metal).
8- Ok, it's now going all the way down, and it's running smooth! The cable is still slack, so that means that we have a true mechanical stop. Almost done!
9- Tighten barrel adjuster to the point where it just goes taught.
10- Rotate the cranks, and go through all the gears.
NOTE- During this process, you might have to loosen the cable clamp to allow slack, as the barrel adjuster might be "bottomed" out. If it is, DO NOT leave it in this position (bottomed out) before you re-tighten the cable clamp. Back it out around one full turn, as this gives you some breathing room for fine tweaks to the system.