Fixing a Corrupt Active Directory Database

Recently I was contacted by a colleague who was having issues with an Active Directory database. Whist there is nothing unusual in this colleague contacting me for help or vice-versa, this issue was beyond the norm.

What he had reported to me was that there was issues with the primary domain controller (PDC) and secondary domain controller (SDC) on this site having out of sync databases, which came to the fore as he was adding new devices (through WDSUtil) to be imaged, they appeared on the SDC but not on the PDC, with this causing issues predominantly with the fact they would image the machine, and get the correct name from the SDC which was also acting as the (Windows Deployment Services) WDS server but it would not bind to the domain, as there was no account for it on the PDC.

Upon further investigation (over the phone at this point) we discovered the the two domain controllers were out of sync and the tombstone had exipred, fixing this problem allowed for a partial sync as outlined below;

PDC==>SDC – Success
SDC==>PDC – Fail

PDC ==>SDC – Success
SDC==>PDC – Success

These tests were run from the “Active Directory Sites and Services” tool on the domain controllers as shown above.

Looking at the error logs it showed AD Domain Services errors of 1988  and an error stating

Active Directory Domain Services Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory Domain Services database. Not all direct or transitive replication partners replicated in the deletion before the tombstone lifetime number of days passed. Objects that have been deleted and garbage collected from an Active Directory Domain Services partition but still exist in the writable partitions of other DCs in the same domain, or read-only partitions of global catalog servers in other domains in the forest are known as “lingering objects”

It did also give a whole bunch of sensitive information (hence I will not publish it) stating the object that was causing it. Looking for the cause of the error I came across the repadmin (AD Replication Admin command line tool)- repadmin /removelingeringobjects ServerWithLingeringObjects CleanServerGUID NamespaceContainingLingeringObject which I ran, and I ran the replication tests again and got the same results.

So figuring I had nothing to loose I deleted the object that was referenced in the error, which in my case was a user, so I do this and try the replication again. This time I got an error stating that “An internal error occurred”, great what next. Looking at the error logs again (on the PDC, as by this time I was pretty sure it was the PDC that was causing the issues) I found an error of 467 meaning a corrupt database…. Oh SHIT… ok not that bad really but still.

I decided that I would try to repaid the database directly rather than using ADRM on the server (as I only had remote access). I stopped the Active Directory Domain Services – service in the Services Manager (services.msc) and knowing that the AD database is a JET database and that it is stored in C:\Windows\NTDS (NTDS Stands for NT Directory Services) I copied the file ntds.dit (the AD Database itself) to the desktop twice (two different file names, one to work on one to back up)

So once I had the two files I ran a verify on the database through the command esentutl /g C:\Users\<USER>\Desktop\ntds.dit the results coming back that the database is in fact corrupt so I ran the fix esentutl /p C:\Users\<USER>\Desktop\ntds.dit I then moved the fixed file back to C:\Windows\NTDS, restarted the Active Directory Domain Services – service in the Services Manager (services.msc) ran the replication tests again, and they all passed

Crisis averted, and I am now owed a good bottle of Scotch Whisky

This was all done over a remote session so it is possible


EMCO Remote Shutdown and Setting Windows 8(.1) Remote Registry by Group Policy Object (GPO)

As I have mentioned in a previous blog post, several clients who have been using this software for several years with their fleets of Windows 7 desktops with great success. This however changed when testing during the Windows 8.1 deployment we found that it does not work for 8/8.1 this is due to the Remote Registry service no longer being enabled by default.


Now rather than wanting to update the machines manually or to change the service status in the image, I wanted to start this service as this will ensure that all devices turn it on and when I or someone else creates a new image in future, it is one less thing to do. It turns out this is easier to do than I thought it would be.

First you need to open up “Group Policy Management“, find the policy you want to edit by expanding the appropriate trees (or create a new policy within the right scope), right click on it and select “Edit“. This is a computer policy so if like me you limit your GPO’s to work on only users OR computers (Best Practice), then make sure you select a computer enabled policy.



Once you have opened the “Group Policy Management Editor” then you will need to navigate the tree (in the left hand column) to “Computer Configuration” > “Policies” > “Windows Settings” > “Security Settings” > “System Services” and then in the right hand column search out “Remote Registry“, double click on this to open the “Remote Registry Properties” box.


In this box, select the “Define this policy setting” checkbox, which will then in turn enable the options below it, and you simply want to change the “Select service startup mode” radio buttons system to “Automatic

Now after a group policy update (which can be forced on individual machines via “gpupdate /force“, without the quotes) and a reboot, the machines will have the “Remote Registry” started and running




EMCO Remote Shutdown

Remote Shutdown from EMCO Software is a great piece of software for helping to manage fleets of Windows Based PC’s in large environments, it uses Wake on LAN to start PC’s up at a certain time, certainly nothing fancy with that, however what else it can do is force log offs and shutdowns on a schedule as well. It does this through some clever use of facilities already built into Windows, but little used. I would certainly rate this software highly and recommend its use to anyone with large fleets to manage.

I have used it at several clients to manage their fleets to start up the PC’s before workers get there, and to shut them down after they leave, this is to stop people leaving them on from sheer forgetfulness (if it needs to be left on we can exclude the PC from shutdown for the required time, but people do need to tell us) and we use it as part of our environmental program to minimize power wastage. People if logged in and using the PC at the time the shutdown or log off instructions come in, can cancel it themselves (we do not want to stop people working now do we).

This has helped with several things, as we have a specified lunch period at each site, I shut down the PC’s 10 minutes after it has started, this is to allow updates to install, and restart the PC’s 5 minutes before people are due back at work. We have a much greater patch ratio now, than before this happened. The schedules at the sites are simply, start at 0800, shutdown at 1345, restart at 1455, shutdown some PC’s at 1610PM, shutdown all PC’s at 2000.

In addition to this when one of the clients had an environmental audit (they were chasing a 6 star environmental rating) the auditor was impressed with the technology and it aided in their gaining of their 6 star rating

All in all I am very impressed with the EMCO solution and highly recommend it

SafeDuino – Part 04: Further Afield – A Complete Redesign?

After having not had much time to work on this project in the past few months, with a major half-million dollar project going on at one of my clients, which was/is the implantation of a new computer system of my own design that will be serving them for the next 20 years or more (well the physical infrastructure will be, the PC’s wireless, network, servers etc won’t they are replaced on a 3 year cycle, again on my insistence) and with one of the suppliers delivering critical components in excess of week late it has pushed back the final deployment for a couple of months as now I can only do core works on weekends and after hours, I have not had much time for this project.

What I have been doing/contemplating is two things however, firstly is whether I need an LCD screen/buttons interface, and I have decided that I do need one, this has become evident so that I can set up the system “in place” when it is finally deployed.

The other thing I want to do is be able to deploy some sensors and control circuits remotely, including some in hard to access places, well ones that once they are in they will not be easily accessible. To this end I have come up with the idea of using network cable for limited power (5VDC for system power) and command and control signals, and where required I will use Flat-Flex Cables’ (FFC’s)  to get into those tight spots that I cannot otherwise support sensors in. 

 What this leaves me with is a system that is essentially divided into three distinct parts, the main controller (the Freeduino EtherMega) the distribution node which is connected to the EtherMega via network cable, and the sensors and control points are connected to the distribution nodes via CAT3 (telephone) cabling or via FFC’s (which are limited to 18″) where required.

This now leaves me of working out how to select which pins go to which distribution point via the network cabling, this I am still trying to solve, however jumper wires are looking like the best option at this point

Product Review: Rifle Rods

I have been thinking about doing this review for a while, but had not got up to doing it…. until now.

As you may have inferred from previous posts, I am a shooter, and considering where I live (Australia, for those who have not worked it out from the domain) I do have a rather extensive, and expensive collection of longarms (well for Australia at least), and as with any hobby, space is always at a premium. Due to this, and the fact I have plans to double my collection over time, I had to find space to fit more into the (do more with less anyone??). Considering I already had two safes, one being used to store ammunition and the other the firearms themselves, as to comply with Australian law, I had to get as much into the Hy-Skor (Hyskor) HSCH2 (a 30 longarm safe) as possible. As anyone who has had a firearms safe with any type of modern scoped rifle in it, can attest there is no way you can fit the stated capacity in the safe as it is (if I am to believe what I am told) based on how many shotguns, specifically over/under shotguns it can fit, not rifles.

So it is with this starting situation I start looking for solutions, as it happens someone else at the time over at (No longer a forum) was looking for something called Rifle Rods (available through Store More Guns) them, and I saw the thread and my interest was piqued. After some research I decided to purchase some on my trip to the US in December 2012/January 2013, and this is exactly what I did, I purchased them and had them delivered to my hotel in San Francisco, then flew back with them.

After several months of having other things to do, including hours of catch up work for taking 6 weeks off (is it really a holiday if you come back and do the work anyway?) I finally got around to doing the first part of the install in April. The first part is/was the modifying of the safe itself, whilst strictly not required and I could have put the special hook and loop material onto the bottom of the shelves and internal compartment, I would have only got about two thirds of the possible shelf space. So instead I took measurements using the pre existing mounting points for the firearms and the pre-existing shelves and internal compartment bottoms to get dimensions for shelves, these turned out to be as follows;

Shelf: 590mm in width; 360mm in depth with 2 folds on the 590mm edge adding another 10mm for each fold, and the material being 1.5mm steel
Internal Compartment Bottom Extension: 540mm in width; 360mm in depth with 1 fold on the 540mm edge adding another 10mm for the fold, and the material being 1.5mm steel

Which I then took to a local metalshop and got them to fold me the new larger shelves, when this was done both shelves were then primed with 2 coats of Kill Rust primer that I purchased from the local hardware store, as well as a top coat of “hammer tone” black to make it more closely match the powder coating of the safe. I also cut and fitted the Hook and Loop material to the bottom of the shelf and the bottom extension, with the shelf also getting a piece of non slip matting being cut for its upper part. That was the easy part done, now to install them.

To install the shelf was the easy part of the installation process, simply take the old one out, and put the new one in, the extension to the internal compartment bottom was a whole different ballgame.

To affix this new bottom/shelf to the safe I firstly had to remove the lining from under the compartment, this was very, very well adhered to the bottom of the compartment with some form of glue, so after about an hour of scraping away at it will a scraper, and making a mess in general but I eventually got a clean(ish) surface to work with, I then proceeded to use SikaBond to glue it to the base, using various objects from around the safe area to hold it in place until the SikaBond had dried, then plus a few days to ensure it had cured.

Once this was done it was simply a matter of re-installing the lighting system and putting the rifles and other bits back in. Now I have not only a safe that will hold the advertised amount, but more than the advertised amount I believe, but I will have to find out at some point.

Overall I am very happy with this, I do want to make a couple of modifications to the system, namely for the shotguns where the rods really need a rubber or silicon part attached to the rods that makes them a little wider so they sit properly upright, I suppose I had better break out the moulding gear and the vacuum chamber.

Now back to the Arduino Project.

SafeDuino – Part 02: (Partial) Parts test

I have now got some of the equipment for my SafeDuino, as has been mentioned previously I have selected Freetronics for much of my kit for this, sure I could make most of it manually, but I do prefer it being on a circuit board to make it that bit easier to mount.

So far I have obtained the following from a local Jaycar store

  • Freetronics EtherMega
  • RGB LED, This may yet be changed, as something I would like to do would be easier if I just had the RGB LED itself without the board, well rather easier without the IC on it, the board is prefered however.
  • Humidity sensor, Freetronics board mounted, however it is a “common” DHT22
  • Reed Switch, this came from my stash of components, but was originally from Jaycar as well
  • 2000 Ohm and 8000 Ohm resistors to make a voltage divider
  • NDriver Transitor relay, this is to allow me to turn on/off the LED light strip

Good news is all parts have so far passed testing, now its just a matter of waiting for the rest of the items to arive, in the mean time however I will not be slacking I will be using a simulator  (Simulator for Arduino from Virtronics) and the components already at hand to start work on this project.

Whats worse is I have already decided on my next project, several (as we have several locations on the property that require it) water tank level sensors, based upon the Freetronics EtherTen, this one has been chosen for that project as I can use the Cisco POE switches I have at home to power it from the network, meaning no external power requirements

%d bloggers like this: