Is Django admin difficult to customize?
You are not providing enough details on what you want to achieve, so it's difficult to say how complex the task is. You might also want to consider not modifying the admin site at all and building your own views where appropriate.
However, here are some good links to get you started:
Personally, if you want a site to look like the admin, why not pull the templates and styles and use them, build your own views for what you need. Gobs of documentation and forum help is there for that. I like the idea of customizing the admin, but honestly, I have been doing it for awhile on a project and time and time again I think to myself, if this was built in the standard MVC (or MTV) manner with free templates online, copied admin ones, or some professionally made ones, and built with the plethora of addons and my code, it would be much easier!!! And, when you decide that request/response isn't cutting it, and you want to add lots of JavaScript, you'll be glad. I know. I have had to put all sorts of JavaScript on our Admin project. Mostly because it's impossible to make the admin do what we want, so we fix it with JavaScript once it is on screen. When you find yourself writing an Ajax based system, you'll wonder why you have the admin at all.
If I could start this project over, I might not even use Django, but I probably would. I most certainly won't used the Admin.
Now, if you are building an basic CRUD type site that doesn't have style-eyed users, then Django with grappelli, and some elbow grease will get the job done. Remember, Django is a collection of Python scripts. Just override everything you can, and you'll get there, if you aren't too ambitious.
Django Admin is easy to customize if your requirements match what is customizable in Django. If you need fancy features that are not out-of-the-box functionality you either need to find a plugin that does it, or implement it yourself by subclassing the appropriate classes. Which can be difficult, and requires good understanding of the internals of Django.
So in summary: it makes it easier to setup an admin, but if you need anything special that's not out of the box: you got a steep learning curve.
Depending on your requirements you can choose to use django or not. Anything that requires a lot of functional speccing is better of implemented manually.