Fixing a Corrupt Active Directory Database

Recently I was contacted by a colleague who was having issues with an Active Directory database. Whist there is nothing unusual in this colleague contacting me for help or vice-versa, this issue was beyond the norm.

What he had reported to me was that there was issues with the primary domain controller (PDC) and secondary domain controller (SDC) on this site having out of sync databases, which came to the fore as he was adding new devices (through WDSUtil) to be imaged, they appeared on the SDC but not on the PDC, with this causing issues predominantly with the fact they would image the machine, and get the correct name from the SDC which was also acting as the (Windows Deployment Services) WDS server but it would not bind to the domain, as there was no account for it on the PDC.

Upon further investigation (over the phone at this point) we discovered the the two domain controllers were out of sync and the tombstone had exipred, fixing this problem allowed for a partial sync as outlined below;

On PDC
PDC==>SDC – Success
SDC==>PDC – Fail

On SDC
PDC ==>SDC – Success
SDC==>PDC – Success

These tests were run from the “Active Directory Sites and Services” tool on the domain controllers as shown above.

Looking at the error logs it showed AD Domain Services errors of 1988  and an error stating

Active Directory Domain Services Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory Domain Services database. Not all direct or transitive replication partners replicated in the deletion before the tombstone lifetime number of days passed. Objects that have been deleted and garbage collected from an Active Directory Domain Services partition but still exist in the writable partitions of other DCs in the same domain, or read-only partitions of global catalog servers in other domains in the forest are known as “lingering objects”

It did also give a whole bunch of sensitive information (hence I will not publish it) stating the object that was causing it. Looking for the cause of the error I came across the repadmin (AD Replication Admin command line tool)- repadmin /removelingeringobjects ServerWithLingeringObjects CleanServerGUID NamespaceContainingLingeringObject which I ran, and I ran the replication tests again and got the same results.

So figuring I had nothing to loose I deleted the object that was referenced in the error, which in my case was a user, so I do this and try the replication again. This time I got an error stating that “An internal error occurred”, great what next. Looking at the error logs again (on the PDC, as by this time I was pretty sure it was the PDC that was causing the issues) I found an error of 467 meaning a corrupt database…. Oh SHIT… ok not that bad really but still.

I decided that I would try to repaid the database directly rather than using ADRM on the server (as I only had remote access). I stopped the Active Directory Domain Services – service in the Services Manager (services.msc) and knowing that the AD database is a JET database and that it is stored in C:\Windows\NTDS (NTDS Stands for NT Directory Services) I copied the file ntds.dit (the AD Database itself) to the desktop twice (two different file names, one to work on one to back up)

So once I had the two files I ran a verify on the database through the command esentutl /g C:\Users\<USER>\Desktop\ntds.dit the results coming back that the database is in fact corrupt so I ran the fix esentutl /p C:\Users\<USER>\Desktop\ntds.dit I then moved the fixed file back to C:\Windows\NTDS, restarted the Active Directory Domain Services – service in the Services Manager (services.msc) ran the replication tests again, and they all passed

Crisis averted, and I am now owed a good bottle of Scotch Whisky

This was all done over a remote session so it is possible

Justin

EMCO Remote Shutdown

Remote Shutdown from EMCO Software is a great piece of software for helping to manage fleets of Windows Based PC’s in large environments, it uses Wake on LAN to start PC’s up at a certain time, certainly nothing fancy with that, however what else it can do is force log offs and shutdowns on a schedule as well. It does this through some clever use of facilities already built into Windows, but little used. I would certainly rate this software highly and recommend its use to anyone with large fleets to manage.

I have used it at several clients to manage their fleets to start up the PC’s before workers get there, and to shut them down after they leave, this is to stop people leaving them on from sheer forgetfulness (if it needs to be left on we can exclude the PC from shutdown for the required time, but people do need to tell us) and we use it as part of our environmental program to minimize power wastage. People if logged in and using the PC at the time the shutdown or log off instructions come in, can cancel it themselves (we do not want to stop people working now do we).

This has helped with several things, as we have a specified lunch period at each site, I shut down the PC’s 10 minutes after it has started, this is to allow updates to install, and restart the PC’s 5 minutes before people are due back at work. We have a much greater patch ratio now, than before this happened. The schedules at the sites are simply, start at 0800, shutdown at 1345, restart at 1455, shutdown some PC’s at 1610PM, shutdown all PC’s at 2000.

In addition to this when one of the clients had an environmental audit (they were chasing a 6 star environmental rating) the auditor was impressed with the technology and it aided in their gaining of their 6 star rating

All in all I am very impressed with the EMCO solution and highly recommend it

SafeDuino – Part 04: Further Afield – A Complete Redesign?

After having not had much time to work on this project in the past few months, with a major half-million dollar project going on at one of my clients, which was/is the implantation of a new computer system of my own design that will be serving them for the next 20 years or more (well the physical infrastructure will be, the PC’s wireless, network, servers etc won’t they are replaced on a 3 year cycle, again on my insistence) and with one of the suppliers delivering critical components in excess of week late it has pushed back the final deployment for a couple of months as now I can only do core works on weekends and after hours, I have not had much time for this project.

What I have been doing/contemplating is two things however, firstly is whether I need an LCD screen/buttons interface, and I have decided that I do need one, this has become evident so that I can set up the system “in place” when it is finally deployed.

The other thing I want to do is be able to deploy some sensors and control circuits remotely, including some in hard to access places, well ones that once they are in they will not be easily accessible. To this end I have come up with the idea of using network cable for limited power (5VDC for system power) and command and control signals, and where required I will use Flat-Flex Cables’ (FFC’s)  to get into those tight spots that I cannot otherwise support sensors in. 

 What this leaves me with is a system that is essentially divided into three distinct parts, the main controller (the Freeduino EtherMega) the distribution node which is connected to the EtherMega via network cable, and the sensors and control points are connected to the distribution nodes via CAT3 (telephone) cabling or via FFC’s (which are limited to 18″) where required.

This now leaves me of working out how to select which pins go to which distribution point via the network cabling, this I am still trying to solve, however jumper wires are looking like the best option at this point

SafeDuino – Part 02: (Partial) Parts test

I have now got some of the equipment for my SafeDuino, as has been mentioned previously I have selected Freetronics for much of my kit for this, sure I could make most of it manually, but I do prefer it being on a circuit board to make it that bit easier to mount.

So far I have obtained the following from a local Jaycar store

  • Freetronics EtherMega
  • RGB LED, This may yet be changed, as something I would like to do would be easier if I just had the RGB LED itself without the board, well rather easier without the IC on it, the board is prefered however.
  • Humidity sensor, Freetronics board mounted, however it is a “common” DHT22
  • Reed Switch, this came from my stash of components, but was originally from Jaycar as well
  • 2000 Ohm and 8000 Ohm resistors to make a voltage divider
  • NDriver Transitor relay, this is to allow me to turn on/off the LED light strip

Good news is all parts have so far passed testing, now its just a matter of waiting for the rest of the items to arive, in the mean time however I will not be slacking I will be using a simulator  (Simulator for Arduino from Virtronics) and the components already at hand to start work on this project.

Whats worse is I have already decided on my next project, several (as we have several locations on the property that require it) water tank level sensors, based upon the Freetronics EtherTen, this one has been chosen for that project as I can use the Cisco POE switches I have at home to power it from the network, meaning no external power requirements

Hyper-V Fix Time Sync issues

I know this has been done to death, but as this is my Blog, and the original idea for it was for me to put all the odds and sods of knowledge in one location so  I did not have to remember every little command, I am doing it again.

Hyper-V on Server 2008 and 2008 R2 has a known issue with time slipping slipping slipping into the future (sorry Steve Miller Band moment there) when using a Hyper-V based Primary Domain Controller (PDC). The first part of this is an east step, you turn OFF “Time Synchronisation” for the PDC, or whichever server takes care of your time syncing on the network (although I do it for all servers) on the Hyper-V host, this is done by selecting the Virtual Machine in the Hyper Visor, opening its properties, selecting integration services and unchecking “Time Synchronisation” as shown in the image below

Virtual Machine Settings - Integration Services - Turn off Time Sync
Virtual Machine Settings – Integration Services – Turn off Time Sync

Secondly to that, on the PDC you should set a known reliable time source, I normally select one from http://pool.ntp.org.

To add this sever and set it to your PDC time server open an Administrative Command Prompt and enter the following commands

net stop w32time
w32tm /config /manualpeerlist:PEERS /syncfromflags:manual /reliable:yes /update
net start w32time

Where PEERS is the selected time server or time server pool.

This should update itself instantly, and keep itself updated

My Thoughts: Fleetio

First of all, what is Fleetio? Fleetio is a web based vehicle maintenance system aimed primarily at companies that manage fleets of vehicles, which is where it got its name… strange about that. This is not to say that it is not useful for us “regular” people who have multiple vehicles, or even one vehicle, it really does handle them all, but it does shine and is primarily designed for those with the large fleets. That’s enough of that, on with my thoughts.

Now why would I be interested in this solution, well apart from being a sole trader, so tracking my vehicles expenses is to say the least very important to me so I can claim the maximum amount back from the tax office, I also have multiple personal vehicles, including motorcycles, work vehicles and a convertible all that need to have maintenance and other expenses tracked to ensure its all kept up to scratch, and having complete logs when you go to sell it certainly helps.

I have been using Fleetio now for several months, well its getting close to 12 I would think and I can honestly say I love it, its not perfect, nor does it do everything I need/want it to do but it is honestly the best solution I have used.

The prices for the service range from free for one user and five vehicles, this is suitable for many Sole Traders and families, up to $149 (USD) a month for 250 vehicles and unlimited users, personally I simply use the free one at this point, although I am considering using the $9.00 in future to add the other vehicles in the family, including several collectors vehicles, a forklift and a boat, this will be a true indication of how well it runs when I do this, but at least all the records are in one place.

Fleetio has many features, many I am simply not going to cover as I have not used them, and if you want to know more about them I suggest you head over to the website (http://www.fleetio.com) and check them out. The features I currently use are; Service Logs, Service Reminders, Fuel Logs, Renewal Reminders and Documents. As I said however there are many other features including reports that can be useful but you really need to check them out for yourself as I don’t use them, but for the features I do use, here is a rundown

  •  Service Logs: As the name suggests, this is where you log the work you have done to your vehicle, be it an addition such as driving lights, or maintenance such as the regular oil change, it records what was done, who it was done by, the date it was done and what it cost, all the relevant details, but there is one thing that I do wish it had, the ability to attach one, or preferably multiple files to the service, things such as receipts, quotes, warranties etc. The system handles that under the documents function, which I will get to later in the article, so why not add the ability to link a document, or as I said preferably multiple documents to a service.
  • Service Reminders: Again as the name suggests this allows for the setting of reminders about services, dependent on the way the vehicle is tracked it is done through Running Time, Hours or Date based periods, for example one of my service intervals for one vehicle is 5000KM or every 6 months, what ever is first. In my case this is set up as an “Oil Change” service which to me is a basic service of oil, filter, grease points etc. This feature also allows you to have subscribers that receive emails when something is about to become due (this warning activates at a pre-determined time prior to the event happening, in the case of the service I have set this to 500KM/2 Weeks) it also emails you until you have completed it, if its overdue.
  • Fuel Logs: Basically the same as service logs, tracks (depending on settings I am using metric) Fuel input, tripmeter reading, cost and cost per unit, with this data it can calculate fuel economy and show you on graphs what is happening, a very useful feature, but it could be made better as I will discuss further on in the article
  • Renewal Reminders: Well what can I say, its a reminder, it reminds you of something, in my case I have commonly set (in addition to service reminders above, these are a different item all together) Registration and Insurance, it also handle inspection and emissions test reminders.
  • Documents: This function allows you to upload documents, any type of documents you need, nothing special, useful but not as useful as being able to attach them to a service or fuel entry, that’s not to say I don’t need to add “normal” documents such as manuals to the information, but I also need the ability to link a document, or as I have previously said, multiple documents to a service and fuel entry, service in particular, but fuel entry would also be useful

Now as I said above,  I am not saying Fleetio is perfect, but it is the best that I have used so far, and I have used quite a few programs for this kind of thing. Firstly if I were over there I would add an API and application interface, now this does not sound like much but there are several benefits and things that could be done with this that can not be done with a website only product, the first is cashing of data, now whilst I am not the same as everyone and I realise and accept this, in my situation even though I predominantly work in IT, I am not always connected to the internet, allowing me to store data locally especially the fuel log would allow me to enter data on the spot, then when the device gets an internet connection and can establish a link back to the Fleetio site it can upload the newly added or changed data.

Secondly and this is the other major change/addition I would make to the service is that of a trip log, this would simply be starting KM/Miles (automatically filled in from current trip reading/end of last trip reading) and ending KM/Miles. This would allow me to keep a log for tax purposes as well as anything else that I might require it for, this kind of data is better cashed in an application and uploaded when available than having to have an internet connection all the time, having said that, even adding this to the web interface would be a good start. To further expand on this if they wanted, they could offer GPS logging of trips, these files could then be automatically uploaded to the website when a connection was available, and be overlayed on a Google (or other) map, and the GPS file available for download if required. To add further function to the trip log, a simple “Toll Cost” field could be added so that the cost of toll road(s) could be recorded for each trip as well

Over the period I have been using Fleetio, I have made several suggestions, some have been acted on, some not. However each and every time I have gone out of my way to help them, (for me this is a big thing as I do not waste time on things that are not useful to my family, friends or myself as it serves no purpose) I have got a response saying they will look into it, they will add it or they do not plan to add it at this stage etc. even getting this response as a NON paying service user means I have no hesitation recommending them to others.

I urge people to check it out, its one useful site I just had to share

Justin

%d bloggers like this: