AAD Connect Error, Encryption Keys Could Not Be Accessed

Hello again,

This installment I have a write up on an error a co-worker encountered in AAD Connect. All credit for write up goes to an engineer by the name Cody Rowe, reposted here with permission. Thanks Cody!

Fixing AAD Connect Service Account Issues

Friday, February 10, 2017

12:58

Issue: Server had not been restarted for a very long time.  Customer needed to install updated VMWare tools for their backup solution.  On reboot, the ad sync service could not start and gave off this error.

Resolution:

That led me to a few thoughts after doing some searching. 

·         An update was installed after reboot.  Maybe this broke the service.

·         The local service account used to start the ad sync service was affected by the reboot.

After coming up empty with repairing, reinstalling and looking over event viewer longer, I decided to proceed with the assumption that the local service account had its password reset.  I reset the password to the local service account :

Added the service account to local administrators to be able to log into the machine with said account.  Logged out and logged in as the service account.  Make sure you update the password associated with the service

Now I went through the process of abandoning the previous encryption key and adding in a new one.  Run miiskmu.exe located in the Bin folder for the ad sync directory with elevated permissions.

You'll want to select Abandon key set, then provide the credentials for the local service account.

When the operation is complete, stop the ad sync service since it will automatically start and attempt to create new encryption keys.  Go back to the key management utility and select add new key to key set.  I opted to select re-encrypt as well.

When that finishes start the service and open powershell.  Run the following.

Import-module adsync

Start-adsyncsyncCycle -policytype delta

After this ran I was getting some errors I did not expect to happen when it was healthy.

Specifically the stopped-extension-dll-exception is what I'm concerned about.  Going to event viewer it appeared that I was getting password issues with the service accounts.

I reset passwords for all the service accounts (AD for both forests, Cloud service account for machine) and added the updated passwords to the connection credentials for all 3 connectors.

After that I ran another delta sync, and everything came back clean.

Editors Note, From another colleague

To add to that, when you abandon the key, all of the data inside the sync database that was encrypted with the old key is invalid.  This includes the passwords for the accounts used by each connector.  This is why the last step was required

Sync Any Folder To OneDrive

I recently started using OneDrive more. Mostly because our company made the switch internally to OneDrive for business and I got that sweet, sweet 1TB of storage.

Personally, at home anyways, I am a google drive user. One of the great features about Drives sync client is the ability to sync any folder of your choosing to their cloud.


By default with OneDrive it only syncs what is in the OneDrive folder. This path to the OneDrive folder can be changed but by default it is located at C:\Users\%USERNAME\OneDrive-$Businessname

Now there is no true way to sync custom folders but there is a work around, a way to trick OneDrive by using Junction Links. A Junction Link appears as the full folder structure to the folder it lives inside but is actually a link to another location. In the screenshot below you can see I have linked my whole "My Documents" folder to OneDrive (click for larger image)

To create this junction link is a simple set of CMD commands. Remember to run as admin. You want to run the command as follows

mklink /j "C:\Users\(your user profile)\OneDrive - Business\Name of junction folder" "C:\source folder path"

The Junction Link does not have to have the same name as the source folder. In the screen shot below you can see the command I ran to place "My Documents" inside of OneDrive.

I hope this helps you to unlock some of the potential of OneDrive. Please note that as far as I can tell this is not a supported solution to what appears to be a blatant lack of a modern feature, proceed with caution, please consult your physician before using, do not operate while impaired, additional taxes and fees may apply, etc...etc...

Exchange Online OWA Redirection

Been awhile! 

In this post I want to talk about Exchange Online and OWA redirection for user who have a mailbox still on premise.

When you set up your hybrid environemt the wizard will automatically set your on premise TargetOwaURL to https://outlook.com/owa/domainname. You can see this by doing a get-organizationrelationship | FL in the shell.



This is the link that a user who has been moved from on premise Exchange into Exchange Online sees when they try to access their mailbox through the on premise OWA url.



This may not be set by default in your tenant. If you do a
get-organizationrelationship | FL 
in Exchange Online you may see a blank value. This is where you can add your onprem OWA url if you see fit so users who are onprem and try to access the cloud OWA will be redirected.

There is a catch though. If the on prem user IS NOT licensed for exchange online in the cloud then this redirect URL will not show up, they will only receive the " :-( Something Went Wrong" screen. See below for examples.



VS



So even if you have a few fully onprem users and wish to use this feature you are still going to have to pay the piper for a license.

Sites and Microsoft Exchange Active Directory Topology Service

Ran into a weird issue today. The services on my labs Exchange server stopped working. When you try to restart them they error out with the following.

Now checking the dependencies shows that most Microsoft Exchange Services rely on the Microsoft Exchange Active Directory Service. No problem I think so I go to start that service and then I get this error.

After searching for a while and not finding a solution that fixed my issue I began to retrace my steps of what I did in my environment from when Exchange was working and when it went down. One of the things I was doing was deploying an empty test site on my DC.

Now I dont know why an empty test site that was not linked to anything broke my service, but it did. Once I deleted this test site and the test link I was able to start the Microsoft Exchange Active Directoy Topology Service.

Who knew ¯\_(ツ)_/¯.

Deleting an old AAD Connect Service account

Just a little tidbit that had me confused for a second. You can not delete an AAD Connect service account from the tenant portal, you have to do it from within Azure AD. Im sure there is also a method to do this from the shell as well. 

A normal AAD Connect service account will look like this



You can tell which one is the new one because the account is always made with the server name after the Sync_ in this case this account is tied to DC01 from within my org.

You also can no longer disable AAD Connect from the new portal from within the GUI. The new portal requires you to connect via Powershell and run

Set-MsolDirSyncEnabled –EnableDirSync $false

Decommision Exchange 2010 and Arbitration Mailboxes

Arbitration mailboxes are where messages awaiting moderator approval are stored, the OAB is generated, and various other administrative tasks. You can not uninstall 2010 while these are still on the database.

When you try to remove the first DB in Exchange 2010, there are a few arbitration mailboxes that will prevent database deletion. You have the choice of moving, removing, or mail-disable these mailboxes. Since you cannot see these in the Exchange Management Console, you need to launch Exchange Management Shell (EMS)

Get-Mailbox -Arbitration

This will list the arbitration mailboxes. To narrow it down to a specific database, you can edit this to:

Get-Mailbox -Arbitration -Database DB1

Once you have your get- command returning the correct list of mailboxes, it's time to move, disable or remove them. Disabling the last arbitration mailbox is not allowed, so I recommend moving them as the first preference here.

Move: Get-Mailbox -Arbitration -Database db1 | New-MoveRequest -TargetDatabase db2

Disable: Get-Mailbox -Arbitration -Database db1 | Disable-Mailbox -Arbitration

Remove: Get-Mailbox -Arbitration -Database db1 | Remove-Mailbox -Arbitration -RemoveLastArbitrationMailboxAllowed

Once you have these taken care of you should be good to go. if something breaks when you remove these Exchange will recreate them automatically if you rerun

setup.exe /preparead

from the installation media.

Hope this helps someone!

Removing a Server 2003 DC

In preparation for Exchange 2016 I had to remove a 2003 DC from my environment because Exchange 2016 requires a forest functional level of 2008.

There are guides available on how to decommission a Domain Controller so I won't go into that. I just want to touch on a small little error I received and how to fix it.

During my dcpromo portion of the decom I ran into a NETLOGON error. This is due to the machine you're trying to decom is still pointing to itself for DNS. To fix this simply go into your NIC settings and change the DNS to point to the new DC.

Sometimes things are easy I guess!