Very early in my professional system administration career, I was lucky (unlucky?) enough to take over a Mandrake server running an alpha release of Samba 2.2.x that was running as a NT domain Primary Domain Controller (PDC). The system actually ran pretty well, with other Windows NT4 servers attached to the domain and authenticating users to those servers from about 90-100 Windows 95 & 98 workstations throughout the building.
Because the system was running well and working for what we needed, in a production setting, I didn’t tend to tinker with the system, because… it was a production environment. Over time, my employer started to get Windows 2000 machines and I was testing out how to image and deploy Windows 2000 computers. In the process, I was creating new machine names over and over and attaching those to the domain.
One day, as I looking at the lists of users on the Mandrake server, I was noticing all the linux users created through attaching those test Windows 2000 workstations to the domain, which is perfectly normal operation for a Samba NT4 style PDC. Looking at the list of users, I decided to delete all the linux users matching with Window 2000 computer accounts that I knew were gone.
So, I went through and deleted about 10-15 linux users after double and triple checking to verify those accounts weren’t in use and then I deleted those using the command below, which will delete the user account and remove the user’s home directory.
userdel -rf workstation01 userdel -rf workstation02
And that’s where things started going south…. Users weren’t able to log in, printing quit, everything just quit working. Fortunately, the day was finished, so I was able to tell the couple of people working, I was looking at it and I’d have it working by the next morning. I didn’t add what was running through my head which was “hopefully”
So, after about 6 hours of diagnostic work of looking at logs, reading error messages, and keeping the panic down to a minimum, I realized as the Samba service was creating workstation accounts, the Samba configuration was setting the linux workstation account with a home directory of /dev/null Webmin wouldn’t delete /dev/null if it was set as a user’s home directory… would it?
Yes… yes, it would.
So, I had the problem, now to find the solution. After a hour of trying to figure out how to recreate /dev/null and not having any success, I called a friend with more experience to come and take a look. It took him 1 minute to fix the problem and 3 minutes to show me the man page describing how to do it.
rm /dev/null mknod -m 0666 /dev/null c 1 3
We deleted the incorrect normal file /dev/null and used mknod to create the normal, special device file of /dev/null and the moment we did that, the test printer at the end of the work bench started printing out all the queued test prints I had sent it.
I thanked my friend, who had just earned many lunches on me, and spent 15 minutes calming down and called it a night. The Mandrake’s kernel never panicked, but I certainly had for an entire evening.
906 Networks 