Recent Updates Toggle Comment Threads | Keyboard Shortcuts

  • efeldhusen 10:23 pm on March 17, 2017 Permalink | Reply
    Tags: /dev/null, linux, servers   

    A panic with no /dev/null on your Linux server 

    Very early in my professional system administration career, I was lucky (unlucky?) enough to take over a Mandrake server running an alpha release of Samba 2.2.x that was running as a NT domain Primary Domain Controller (PDC).  The system actually ran pretty well, with other Windows NT4 servers attached to the domain and authenticating users to those servers from about 90-100 Windows 95 & 98 workstations throughout the building.

    Because the system was running well and working for what we needed, in a production setting, I didn’t tend to tinker with the system, because… it was a production environment.  Over time, my employer started to get Windows 2000 machines and I was testing out how to image and deploy Windows 2000 computers.  In the process, I was creating new machine names over and over and attaching those to the domain.

    One day, as I looking at the lists of users on the Mandrake server, I was noticing all the linux users created through attaching those test Windows 2000 workstations to the domain, which is perfectly normal operation for a Samba NT4 style PDC.   Looking at the list of users, I decided to delete all the linux users matching with Window 2000 computer accounts that I knew were gone.

    So, I went through and deleted about 10-15 linux users after double and triple checking to verify those accounts weren’t in use and then I deleted those using the command below, which will delete the user account and remove the user’s home directory.

    userdel -rf workstation01
    userdel -rf workstation02

    And that’s where things started going south…. Users weren’t able to log in, printing quit, everything just quit working.  Fortunately, the day was finished, so I was able to tell the couple of people working, I was looking at it and I’d have it working by the next morning. I didn’t add what was running through my head which was “hopefully”

    So, after about 6 hours of diagnostic work of looking at logs, reading error messages, and keeping the panic down to a minimum, I realized as the Samba service was creating workstation accounts, the Samba  configuration was setting the linux workstation account with a home directory of /dev/null  Webmin wouldn’t delete /dev/null if it was set as a user’s home directory… would it?

    Yes… yes, it would.

    So, I had the problem, now to find the solution.  After a hour of trying to figure out how to recreate /dev/null  and not having any success, I called a friend with more experience to come and take a look.  It took him 1 minute to fix the problem and 3 minutes to show me the man page describing how to do it.

    rm /dev/null
    mknod -m 0666 /dev/null c 1 3

    We deleted the incorrect normal file /dev/null and used mknod to create the normal, special device file of /dev/null and the moment we did that, the test printer at the end of the work bench started printing out all the queued test prints I had sent it.

    I thanked my friend, who had just earned many lunches on me, and spent 15 minutes calming down and called it a night.   The Mandrake’s kernel never panicked, but I certainly had for an entire evening.

  • efeldhusen 11:55 pm on March 7, 2017 Permalink | Reply  

    A step on the path 

    I think it’s time to start a blog. I’d like to remember the things I’ve done in technology, what I’m working on now, and what I’d like to explore.

    In time, I expect myself to get a little bit more expressive.

    • Jason Poll 8:27 pm on March 8, 2017 Permalink | Reply


      ….wasn’t that the fashionable thing to do on slashdot and digg? Always the race to be the first commenter? 😉

      Liked by 1 person

      • efeldhusen 8:02 am on March 9, 2017 Permalink | Reply

        Oh, those two references are really carbon dating us. 😀


    • Todd Moser 7:27 am on March 9, 2017 Permalink | Reply


      Liked by 1 person

Compose new post
Next post/Next comment
Previous post/Previous comment
Show/Hide comments
Go to top
Go to login
Show/Hide help
shift + esc