jQuery Sortable without jQuery UI
For future generations: http://dragsort.codeplex.com/
A demo: http://cleesh.org/example (couldn't find one on the development website).
Minified it comes to 9KB in size.
You can build your own JQuery download on jqueryui.com without all the css/theme information. You can also strip out the widgets and effects and just use draggable/droppable.
Comes to about 30KB all in. Still too large?
Here is another one for future generations: http://johnny.github.com/jquery-sortable/.
It's only dependency is jQuery, has a nice feature set and comes in at 8kb minified.