Enabling Data Deduplication on Server 2012 R2


Data deduplication (or dedupe for short) is a process which by the system responsible for the deduplication scans the files in one or more specific locations for duplicates, and where duplicates are found it replaces all the duplicate data with a reference to the “original” data. This in essence is a data compression technique designed to save space by reducing the data actually stored, as well as aiming to provide single-instance data storage (storing only one copy of the data, no matter how many places its located in).

The way this is achieved is dependent on the system used, it can be done but it can be done on block level, file level or other levels, again depending on the system and how it is implemented.

What we are going to do in this article is we are going to enable deduplication on a Windows Server 2012 R2 Server. Keep in mind this is changing data and quite possibly going to cause data damage or loss, as such make sure you have a working backup BEFORE continuing.

Firstly we need to access the server that you are planning to configure deduplication on, I will leave it up to you how you achieve that. Once you have access to the server we can begin.

On the server open “Server Manager” if it is not already open

2014-09-19-01-ServerManager

If it gives you the default splash page, simply click next (and I suggest telling it to skip that page in future by use of the checkbox) Once we are in the “Installation Type” page we need to select “Role-based or feature-based installation” and click “Next”

2014-09-19-02-AddRoleorFeature

In the “Server Selection” page select the server you want to install the service on (commonly the one your using), Click “Next”

2014-09-19-03-SelectServer

 

Next up is the “Server Roles” page, here is where the configuration changes need to take place. In the right had list of checkboxes (titled “Roles”) scroll down till you see “File And Storage Services” then open “File and iSCSI Services” then further down the page check the “Data Duplication” checkbox. Click “Next”, accepting any additional features it wants to install.

2014-09-19-04-SelectService

In the “Features” page simply click “Next”

2014-09-19-05-IgnoreFeatures

On the “Confirmation” page check you are installing what is required and click “Install”

2014-09-19-06-Install

Wait for the system to install, and exit the installer control panel, restart if your server requires it.

Upon completion of the install and any tasks associated with the installation re-open “Server Manager” and in the left hand column select “File and Storage Services”

2014-09-19-07-ServerManager

This will change the screen in “Server Manager” to a three column layout, in the middle column select “Volumes”

2014-09-19-08-ServerVolumes

With the volumes now displaying in the right hand of the three columns, right click on the volume you want to configure deduplication on and select “Configure Data Deduplication”

2014-09-19-09-ServerVolumesRightClick

This will bring up the “Deduplication Settings” screen for the volume you right clicked on. Unless Data Deduplication has been configured before, the “Data deduplication” will be “Disabled”.

2014-09-19-10-DuplicationSettings-Initial

As I am configuring this on a file server, I am going to select the “General purpose file server” option, and leave the rest as defaults. I am then going to click on the “Set Deduplciation Schedule” button

2014-09-19-11-DuplicationSettings-Enable

The “Deduplication Schedule” will now open. I suggest checking the “Enable background optimization” checkbox as this will allow the server to optimise data in the background. I also elected to create schedules to allow for more aggressive use of system resources, the first one allows for it to be done after most people have left for the day, and before the servers scheduled backup, the second one allows it to run all weekend but again stops for backups. Please note that these settings are SYSTEM settings and apply to all data deduplication jobs on the system, and are not unique to each individual deduplication job

Click “Apply” on the “Deduplication Schedule” screen, and then “Apply” on the “Deduplication Settings” screen, this will drop you back to the “File and Storage Services > Volumes” screen, and you are now done, Data deduplication is configured.

Have fun, and don’t forget that backup

Justin

Fixing a Corrupt Active Directory Database

Recently I was contacted by a colleague who was having issues with an Active Directory database. Whist there is nothing unusual in this colleague contacting me for help or vice-versa, this issue was beyond the norm.

What he had reported to me was that there was issues with the primary domain controller (PDC) and secondary domain controller (SDC) on this site having out of sync databases, which came to the fore as he was adding new devices (through WDSUtil) to be imaged, they appeared on the SDC but not on the PDC, with this causing issues predominantly with the fact they would image the machine, and get the correct name from the SDC which was also acting as the (Windows Deployment Services) WDS server but it would not bind to the domain, as there was no account for it on the PDC.

Upon further investigation (over the phone at this point) we discovered the the two domain controllers were out of sync and the tombstone had exipred, fixing this problem allowed for a partial sync as outlined below;

On PDC
PDC==>SDC – Success
SDC==>PDC – Fail

On SDC
PDC ==>SDC – Success
SDC==>PDC – Success

These tests were run from the “Active Directory Sites and Services” tool on the domain controllers as shown above.

Looking at the error logs it showed AD Domain Services errors of 1988  and an error stating

Active Directory Domain Services Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory Domain Services database. Not all direct or transitive replication partners replicated in the deletion before the tombstone lifetime number of days passed. Objects that have been deleted and garbage collected from an Active Directory Domain Services partition but still exist in the writable partitions of other DCs in the same domain, or read-only partitions of global catalog servers in other domains in the forest are known as “lingering objects”

It did also give a whole bunch of sensitive information (hence I will not publish it) stating the object that was causing it. Looking for the cause of the error I came across the repadmin (AD Replication Admin command line tool)- repadmin /removelingeringobjects ServerWithLingeringObjects CleanServerGUID NamespaceContainingLingeringObject which I ran, and I ran the replication tests again and got the same results.

So figuring I had nothing to loose I deleted the object that was referenced in the error, which in my case was a user, so I do this and try the replication again. This time I got an error stating that “An internal error occurred”, great what next. Looking at the error logs again (on the PDC, as by this time I was pretty sure it was the PDC that was causing the issues) I found an error of 467 meaning a corrupt database…. Oh SHIT… ok not that bad really but still.

I decided that I would try to repaid the database directly rather than using ADRM on the server (as I only had remote access). I stopped the Active Directory Domain Services – service in the Services Manager (services.msc) and knowing that the AD database is a JET database and that it is stored in C:\Windows\NTDS (NTDS Stands for NT Directory Services) I copied the file ntds.dit (the AD Database itself) to the desktop twice (two different file names, one to work on one to back up)

So once I had the two files I ran a verify on the database through the command esentutl /g C:\Users\<USER>\Desktop\ntds.dit the results coming back that the database is in fact corrupt so I ran the fix esentutl /p C:\Users\<USER>\Desktop\ntds.dit I then moved the fixed file back to C:\Windows\NTDS, restarted the Active Directory Domain Services – service in the Services Manager (services.msc) ran the replication tests again, and they all passed

Crisis averted, and I am now owed a good bottle of Scotch Whisky

This was all done over a remote session so it is possible

Justin

EMCO Remote Shutdown and Setting Windows 8(.1) Remote Registry by Group Policy Object (GPO)

As I have mentioned in a previous blog post, several clients who have been using this software for several years with their fleets of Windows 7 desktops with great success. This however changed when testing during the Windows 8.1 deployment we found that it does not work for 8/8.1 this is due to the Remote Registry service no longer being enabled by default.

2014-08-11-RemoteRegistry-00-DisabledRegistry

Now rather than wanting to update the machines manually or to change the service status in the image, I wanted to start this service as this will ensure that all devices turn it on and when I or someone else creates a new image in future, it is one less thing to do. It turns out this is easier to do than I thought it would be.

First you need to open up “Group Policy Management“, find the policy you want to edit by expanding the appropriate trees (or create a new policy within the right scope), right click on it and select “Edit“. This is a computer policy so if like me you limit your GPO’s to work on only users OR computers (Best Practice), then make sure you select a computer enabled policy.

2014-08-11-RemoteRegistry-01-GPEDIT

 

Once you have opened the “Group Policy Management Editor” then you will need to navigate the tree (in the left hand column) to “Computer Configuration” > “Policies” > “Windows Settings” > “Security Settings” > “System Services” and then in the right hand column search out “Remote Registry“, double click on this to open the “Remote Registry Properties” box.

2014-08-11-RemoteRegistry-03-EditPolicy

In this box, select the “Define this policy setting” checkbox, which will then in turn enable the options below it, and you simply want to change the “Select service startup mode” radio buttons system to “Automatic

Now after a group policy update (which can be forced on individual machines via “gpupdate /force“, without the quotes) and a reboot, the machines will have the “Remote Registry” started and running

2014-08-11-RemoteRegistry-04-RegistryEnabled

 

Justin

EMCO Remote Shutdown

Remote Shutdown from EMCO Software is a great piece of software for helping to manage fleets of Windows Based PC’s in large environments, it uses Wake on LAN to start PC’s up at a certain time, certainly nothing fancy with that, however what else it can do is force log offs and shutdowns on a schedule as well. It does this through some clever use of facilities already built into Windows, but little used. I would certainly rate this software highly and recommend its use to anyone with large fleets to manage.

I have used it at several clients to manage their fleets to start up the PC’s before workers get there, and to shut them down after they leave, this is to stop people leaving them on from sheer forgetfulness (if it needs to be left on we can exclude the PC from shutdown for the required time, but people do need to tell us) and we use it as part of our environmental program to minimize power wastage. People if logged in and using the PC at the time the shutdown or log off instructions come in, can cancel it themselves (we do not want to stop people working now do we).

This has helped with several things, as we have a specified lunch period at each site, I shut down the PC’s 10 minutes after it has started, this is to allow updates to install, and restart the PC’s 5 minutes before people are due back at work. We have a much greater patch ratio now, than before this happened. The schedules at the sites are simply, start at 0800, shutdown at 1345, restart at 1455, shutdown some PC’s at 1610PM, shutdown all PC’s at 2000.

In addition to this when one of the clients had an environmental audit (they were chasing a 6 star environmental rating) the auditor was impressed with the technology and it aided in their gaining of their 6 star rating

All in all I am very impressed with the EMCO solution and highly recommend it

Field Firing Solutions (FFS) Delta V for Tablets?

As you may have guessed from my other posts I shoot, my interest primarily is long range shooting. In this field the best software in my opinion is Field Firing Solutions (FFS) Delta V, this is what I was trained on it’s use by Glen at Precision Shooting. I do however have one issue with it, this is the requirement for dedicated hardware, specifically a Windows PDA. Now whilst I understand all to well that this makes it easier for development in some regards, I personally would prefer an “App” for either iOS or Windows 8.

There as I see it a number of benefits to this model. firstly most smartphones and PDA’s now offer bluetooth, wireless and in some cases 3G/4G Data connectivity allowing easy connectivity between services, but also internet data, which has never been as readily accessed before, and can provide a whole new level of data input to the device, as well as allowing communications of solutions between teams and commanders in a military or police application.

This brings me to another point, digital distribution. With one of these dedicated hardware devices, they are not only often pricey, they are also more difficult to obtain. If for instance the device breaks just before that yearly hunt or competition (as they always do) then it is not easy to obtain another one quickly, or if in fact you can, cheaply. I mean sure, I could keep a spare one on hand but most people do not a budget that allows them to keep multiple devices on hand.

However, if they for instance used an iOS based device (or Android, or Windows 8 etc.), one could simply go to the nearest stockist (of which there are many, as after all they are consumer devices not specialist devices after all) and purchase a replacement, install the app from the digital distributor and there is your replacement, pair this with a cloud service such as iCloud, Skydrive, Dropbox or one of any of the other numerous options and you can simply get your data back from the cloud, no loss (that is assuming backups are run reguarly, which people do right… ok on second thoughts background syncing would be better) .

There you have it, my thoughts on why Field Firing Solutions should make a iOS/Android/Windows 8 App version of their otherwise wonderful software

Internet Explorer & Other Multi-Tab Group Policy Settings

So visiting a client today and I noticed that the group policy settings for Internet Explorer were not applying correctly, some settings were aplying, some not. After looking at the settings and the RSoP, I decided to take a look at the XML generated for the GPO, and that’s when I saw it… disabled=1, hello. Now the question is where did it come from, I know the GPO is working in other respects, and checking the XML confirms this, so I re-generate the settings, 5 minutes work but still no resolution, its still stating that it is disabled.

What is a IT professional to do, woe is me… Looking further into the situation I came across something I had long forgotten, and filed under “I will never need that”. What was happening or rather not happening was  I was not enabling the fields, specifically in a multi-tabbed setting you will see a red dashed line or a red “No” symbol (the same red border and line that you see everywhere something is prohibited, no smoking sign for example). As shown below

stopped symbol   Dashed Line

To enable the settings you need to hit one or more keys depending on how you want to do it, these keys are as outlined below

F5 – Enable all items on the Page/Tab
F6 – Enable Currently Selected item on Page/Tab
F7 – Disable Currently Selected item on Page/Tab
F8 – Disable all items on the Page/Tab

Once you have enabled the items you will get either a solid green line or a green OK symbol as shown below

green line  go symbol

 Once this was done, I simply forced an update of group policy, and viola, everything worked as it should once more

Moral of this story, its the things you think you don’t need, that ultimately you will need

Hyper-V Fix Time Sync issues

I know this has been done to death, but as this is my Blog, and the original idea for it was for me to put all the odds and sods of knowledge in one location so  I did not have to remember every little command, I am doing it again.

Hyper-V on Server 2008 and 2008 R2 has a known issue with time slipping slipping slipping into the future (sorry Steve Miller Band moment there) when using a Hyper-V based Primary Domain Controller (PDC). The first part of this is an east step, you turn OFF “Time Synchronisation” for the PDC, or whichever server takes care of your time syncing on the network (although I do it for all servers) on the Hyper-V host, this is done by selecting the Virtual Machine in the Hyper Visor, opening its properties, selecting integration services and unchecking “Time Synchronisation” as shown in the image below

Virtual Machine Settings - Integration Services - Turn off Time Sync
Virtual Machine Settings – Integration Services – Turn off Time Sync

Secondly to that, on the PDC you should set a known reliable time source, I normally select one from http://pool.ntp.org.

To add this sever and set it to your PDC time server open an Administrative Command Prompt and enter the following commands

net stop w32time
w32tm /config /manualpeerlist:PEERS /syncfromflags:manual /reliable:yes /update
net start w32time

Where PEERS is the selected time server or time server pool.

This should update itself instantly, and keep itself updated

Global DNS Blocklist

After the rebuild of a AD Domain Controller I was wondering why I could not longer get response from WPAD, when it hit me like a ton of bricks….. Global DNS Blacklist, this is a “feature” in some Microsoft products, and in this case specifically Server 2008 R2 that blocks the query of specific DNS names (isatap and wpad by default, although you can add and remove names to the list), apparently for security so that the address cannot be used to gain unauthorized access to the system through spoofing, all well and good, and I am all for added security but a number of browsers require it for automatic proxy detection, hence we  have to disable it.

Thankfully that is easy enough through an ADMINISTRATIVE command prompt using the following commands

If you want to check that the DNS blocklist is enabeld, type; dnscmd /info /enableglobalqueryblocklist if it displays 1, its enabled, if 0 its disabled nice and simple, but wait what if you want to see the contents of the blocklist, again simple through an administrative command prompt (lets assume from now on in this article that all command prompts are administrative shall we) simply type; dnscmd /info /globalqueryblocklist this will make the blocklist print out onto your screen

Now how to disable it, easy simply input the following commands

  1. dnscmd /config /globalqueryblocklist (Optional, this clears the blocklist that way if something happens and it is re-activated it is empty)
  2. dnscmd /config /enableglobalqueryblocklist 0

The second command there is the one that does the actual disabling, conversely if you want to enable it you should type dnscmd /config /enableglobalqueryblocklist 1. As an asside, if you want to ADD an item to the blocklist this is done by typing the following: dnscmd /config /globalqueryblocklist name where name is the item you want to add to the blocklist.

 

Also dont forget to ensure that the mimetype for the file is defined as “application/x-ns-proxy-autoconfig”

Windows Vista/7 God Mode

Here is a neat trick for those of you who do no know it already, Windows Vista and 7 have a “God Mode”, nothing more really than a way to access the system settings, there are however a few settings that are not normally available apparantly, although I have not looked at the whole list of options myself

 

Just create a folder somewhere with the name GodMode.{ED7BA470-8E54-465E-825C-99712043E01C} (“GodMode” can be replaced with anything you want, its the “extension” that is important)

 

[SOLVED] WSUS Update Error, Not reporting and error 800B0001

After a WSUS Rebuild, I started noticing that Machines, although associating with WSUS were showing up that they had not yet reported to the server, upon investigating this it was discovered that the clients were erroring and displaying error code 800B0001. The machine in question hosting WSUS is a 64 Bit Server 2008 R2 machine, with these details in hand I go off looking for a solution.

Looking for solutions to this I came across several sources indicating that this is a known problem, and thankfully that there is a solution available from Microsoft (http://www.microsoft.com/en-us/download/details.aspx?id=29999)

I simply installed the update, and restarted at then end as asked by the installer, once the server is back up I went back to the same clients and re-ran windows update, and off it went working again.

Nice Simple fix, if only everything was that easy