Starting Exchange Server with a blank Information Store

Starting Exchange Server with a blank Information Store

There are some situations where starting Microsoft Exchange with a blank database may be necessary. In my line of work, I run across a large amount of businesses that have had their Exchange Private Information Store corrupted and the whole organizations Email capabilities halted as well. Quite often, getting the users back up and running takes precedence over getting the data back. Don’t get me wrong; the data is still extremely important, however not having email capability can stop some businesses day to day operations dead in their tracks.

In a perfect world, there would and should be a backup Exchange server just waiting to take over in case of a catastrophic event. But in reality, that is rarely the case. Rather than wait a complete day or two, or even longer, for the systems administrator to get the Exchange database recovered, a viable alternative is to restart the information store with a blank database and import the data back in when it is recovered.

There may be other reasons for wanting to create a blank database as well. You may have an Exchange server that is years old, with tons of residual data from users no longer at the company. You may need to free up disk space on your server. You may have a database with minor corruption and decide to ExMerge your data out and import it back in to a clean corruption-free database. Whatever the reason, make sure you have a complete plan of action and be sure to backup your data in case you run into difficulties. The following article explains how to create a new database with Exchange Server.

To start Exchange Server with a blank Information Store:

1. Locate the Exchange database directory and transaction log directory
   1. Open Exchange System Manager
   2. Navigate to Administrative Groups->First Administrative Group->Servers->servername
   3. Underneath servername click First Storage Group and then Action->Properties
   4. Transaction Log location will be listed on the General tab. Note this location (Image 1d)
   5. Navigate under First Storage Group to your Mailbox Store and click Action->Properties
   6. Click the Database tab to note the Exchange Database and Exchange Streaming Database locations (Image 1f)
   7. Do the same for the Public Store

Image 1d	Image 1f

2. Stop the Exchange Information Store (IS) if it is currently running
   1. Click on Start->Programs->Administrative Tools and then on Services or you can go to Computer Management by Right-Clicking on My Computer and choosing “Manage”

      i. If using Computer Management, drill down to Services and Applications, and then Services underneath that

   2. In the right window of the Services or Computer Management console, locate Microsoft Exchange Information Store
   3. If it’s status is listed as “Started”, Right-Click it and choose “Stop” (Image 2c)
   4. It may give you a message stating that dependency services such as Microsoft Exchange Event will need to stop as well. Choose “Yes” to continue stopping the IS

Image 2c

3. Rename database and transaction log directories and create new ones
   1. Rename the database location MDBDATA directory to MDBDATA-old (Image 3a)
   2. Create a new MDBDATA directory
   3. Rename the transaction log MDBDATA directory to MDBDATA-old (if location is different from the database location)
   4. Create a new MDBDATA directory for the transaction logs (Image 3d)

Image 3a	Image 3d

4. Start the Exchange Information Store service
5. Create new data files
   1. From Exchange System Manager navigate to Administrative Groups->First Administrative Group->Servers->servername->First Storage Group
   2. Click on the Mailbox Store and then on Action->Mount Store (Image 5b)
   3. You will receive a message stating that mounting this store will force the creation of an empty database, choose “Yes” to continue (Image 5c)
   4. The Store should mount, give you a message stating it successfully mounted and the data files should be created in the MDBDATA directory. (Images 5d1 & 5d2)
   5. Follow the same steps for the Public database

 

 

Image 5b
Image 5c
Image 5d1	Image 5d2

6. Test and Verify
   1. Verify the data files were created in the MDBDATA directory
   2. Check the Event Log for any errors
   3. Test connection to the Exchange server from Outlook

Exchange Server Data Recovery Overview

I have been managing DTI Data Recovery’s Exchange Server Support Hotline for over 3 years now. I receive a lot of calls about a lot of different types of Exchange Server problems. We also have a detailed Exchange Server Disaster Recovery Blog that has a lot of information that is helpful to Exchange Administrators.

Exchange Server Disaster Recovery

Most times a problem with Exchange Server is fatal. If your Exchange EDB won’t mount or Exchange services won’t start, believe me, it is a disaster. I have written a few articles about the different types of Exchange problems and what you can do to resolve them.

Exchange Server Support Articles on this blog:

• ESEUTIL and ISINTEG for Data Recovery of Failed Exchange Server - this article is about ESEUTIL and ISINTEG, what they are and how they can help you in a disaster.
• Create a Clean PRIV PRIV1 EDB Database - here I explain how to set up a clean database without re-installing Exchange Server. Very helpful especially if you are doing a mailbox level restore.
• Exchange Server Dirty Shutdown - by far the most common unrecoverable scenario, the dirty shutdown often means restore from backup.
• Exchange Server Disaster Recovery - a general overview of our Exchange Disaster data recovery services.
• FREE Exchange Server Disaster Recovery Support - this article explains how and why DTI Data Recovery offers free exchange server support.
• Using EXMERGE to import PST files - A tutorial on how to use ex-merge to import either recovered or backed up PST files back into your Exchange Server.

If you require Exchange Support call: Toll Free 1-866-438-6932 or direct 1-727-345-9665.

BEFORE RUNNING ANY UTILITIES:
Make a copy of your priv.edb and log files (Exchange 5.5) or priv1.edb and stream files (Exchange 2000 or 2003) and pub.edb! We can recover your email if you make copies prior to running utilities. DO NOT run MS utilities!!!

We offer 24 hour Exchange Support at 727-251-2058.

Visit our main Exchange Server Data Recovery page for more information.

Exchange Server Support on Exchange Disaster Recovery Case Studies Blog:

• Microsoft’s Exchange Server Disaster Recovery Action Plan - this is an actual email sent from Microsoft to one of our customers. It is their disaster recovery plan laid out step by step using ESEUTIL and ISINTEG.
• Using Windows Native Backup For Exchange - John Best’s tutorial with graphics on how to use Native Backup to protect your EDB files. You can use this even if you already have another back up program. You can never have enough backup that is for sure!
• Restoring A Windows Native Backup Exchange Server - John goes on to explain how to restore the backup file in this article, you can’t have one without the other right?

I hope you find these articles useful in your Exchange Server adventures!

Exchange Server Tools: Use ExMerge To Import PST Files

Exmerge is a very usefull tool that can import PST files back into your Exchange Server Information Stores. Here is a tutorial from our Exchange Engineer John Best on how to use ExMerge.

Using ExMerge to import PST files

1.       Place all of the PST files that you want imported into a single Folder

2.       Make sure the Exchange Information Service is started

3.       You need to make sure the account you are logged in with is an administrative account and you need to grant this account the appropriate permissions to read the Exchange Mailboxes.  (By default, all administrative accounts are denied permission.)  You can assign the correct permissions by following the steps below:
a.       Create a Windows Security Group, and name it something such as “Exchange Recovery Administrators”.
b.       Add the Windows account you are using to run ExMerge to this group. This account should already be an Exchange administrator account and have local administrator permissions on the Exchange server(s) involved in the mailbox merge process.
c.       In Exchange System Manager, locate the target database and open its Properties dialog box. On the Security tab, add the Exchange Recovery Administrators group and grant this group Full Control permissions on the database.
d.       It may be necessary to wait up to 15 minutes for the permissions granted to take effect. Alternatively, you can reset cached permissions by stopping and restarting all Exchange services, the IIS Admin Service, and the Windows Management services. Because of this latency, you should grant necessary permissions as soon as you know you will need them, not just before you need to use them.

4.       Copy the files Exmerge.exe and Exmerge.ini to your Exchange Bin directory.  (Default is C:\Program Files\Exchsrvr\Bin for Exchange 2000 and 2003)

5.       Run Exmerge.exe

6.       Click Next on the Welcome screen

7.       At Procedure Selection, choose Extract or Import (Two Step Procedure) and click Next

8.       Choose Step 2: Import data and click Next

9.       On the Destination Server page, enter the name of your exchange server.  You may also enter additional options by clicking on the options button:
a.       On the Import Procedure tab, you have your choice of:
1.       Copy (will copy even duplicates without checking)
2.       Merge (will check for duplicates before copying)
3.       Replace (will overwrite duplicates with data from the PST)
b.       On the Folders tab you can choose folders to skip (deleted items, etc…)
c.       On the Dates tab you can choose messages that meet a date range

10.   After entering the server name and any options, click Next

11.   Mailbox Selection – here you can choose the mailboxes to import, if you select all, any mailboxes that a PST does not exist for will be skipped.

12.   NOTE:  ExMerge will look for a PST file that matches a naming convention of “Alias Name”.PST, when a recovery is performed the Alias name is unknown to DTI Data, so the PST files will be named “Display Name”.PST, which may match your organizations Alias or it may not.  If it does not match what is listed as the alias for a user, Exmerge will skip that mailbox.  You will see in the log files if a mailbox is skipped because of a PST not being found.  It may be necessary to rename the PST files prior to running Exmerge.  For example, on my server here, my mailbox name is

John Best, the alias name is Jbest.  When I recovered the mailbox, the PST file was named

John Best.pst, so was initially skipped by Exmerge.  I had to change the PST file name to Jbest.pst and run Exmerge a second time.

13.   On the Locale Selection, leave the default (English), unless necessary to change it and click Next

14.   Target Directory – Browse to the folder containing your PST files and click Next

15.   Save Settings – if you choose, you may save the settings into an Exmerge.ini file for use at a latter time

16.   After clicking Next, Exmerge will run through the process of importing data, utilizing several threads.  After it is finished, click the Finish button

17.   Check the logs (from the same directory Exmerge was run from) named Exmerge.log and ExMerge-(Thread[n]).log for any errors encountered during the process.

18.   The errors most often encountered are:
a.       [14:25:53] (Thread1)Errors encountered. Copy process aborted for mailbox (This is a standard error message that points you to the particular Thread(n).log file to view what actually happened.
b.       [14:25:53] Error. File ‘E:\EXTEST\USERNAME.PST’ not found. Cannot merge data into mailbox for user ‘USER NAME’ (’USERNAME‘). – This basically tells us a PST file was not found for the particular mailbox.  This can be due to Exmerge looking for ALIAS.pst and the file being named USERNAME.pstc.       [14:28:52] Error opening message store (MSEMS). Verify that the Microsoft Exchange Information Store service is running and that you have the correct permissions to log on. (0×8004011d) – This error is normally due to the account not having the appropriate permissions necessary for Exmerge to open the mailbox.  Follow the steps from number 3 in this document to assign the correct permissions.

19.   Further Information on ExMerge:
a.       Download: http://www.microsoft.com/downloads/details.aspx?FamilyID=429163ec-dcdf-47dc-96da-1c12d67327d5&DisplayLang=en
b.       Documentation:
1.       http://support.microsoft.com/kb/174197
2.       http://support.microsoft.com/kb/265441
c.       ExMerge Strategies and Best Practices: http://technet.microsoft.com/en-us/library/51cd78ab-49c4-4d90-9aa4-29dca171cd31.aspx
d.       Troubleshooting ExMerge issues: http://technet.microsoft.com/en-us/library/bb124178.aspx
e.       ExMerge is your Friend: http://msexchangeteam.com/archive/2004/07/01/171051.aspx
For Help with Exchange Server Data Recovery call DTI Data 1-866-438-6932

ESEUTIL and ISINTEG Exchange Server Data Recovery Options

ESEUTIL and ISINTEG are Exchange Server built-in tools that can help recover corrupt PRIV, PRIV1, Pub, and Pub1 edb and stm database files. It is important that prior to running ANY Microsoft Utilities to back up your latest copies of your Information Store files to another partition, network drive or external hard drive.

If you are using Microsoft phone support, they will not tell you to back up your Information Stores! Before letting anyone touch your server back up your database files!

ESEUTIL Disaster Recovery Options

When your Exchange Server has failed, whether from a dirty shutdown, IS reaching the 16GB limit, or even the partition or hard drive running out of space, you are faced with a disaster recovery scenario. There are several steps that must be taken in a particular order for any chance at Exchange Recovery.

Please back up your databases prior to running any utilities.

The first step that must be taken is ESEUTIL with a p switch. Below are the most common ESEUTIL command line switches:

• Defragmentation: ESEUTIL /d <database name> [options]
• Recovery: ESEUTIL /r [options]
• Integrity: ESEUTIL /g <database name [options]
• File Dump: ESEUTIL /m[mode-modifier] <filename>
• Repair: ESEUTIL /p <database name [options]
• Restore: ESEUTIL /c[mode-modifier] <path name> [options]
• Checksum: ESEUTIL /k <database name> [options]

 As you can see with the list a p switch is the repair. It goes through 3 different tests prior to completion. If you don’t make it through these steps it is time to restore from backup. If you don’t have a backup, you will need to call 727-251-2058 after hours or 866-438-6932 from 9 to 5 EST for Exchange Server Disaster Recovery Support. More than likely you will need data recovery.

If the /p does go through its steps without error you will need to remove ALL log files out of your Information Stores folder (usually MDBDATA) examples are edb.log or edb1234.log. At this point you are ready to run a defrag.

ESEUTIL Offline Defrag /d Switch

While it is true most administrators of Exchange schedule regular defrags, you absolutely must run one during Exchange Data Recovery. The length of time that the defrag takes depends on the amount of garbage or “white space” within the edb and your hardware. The older and slower your box, the longer it will take!

Offline defrag creates a new database by copying all records and tables from the old database into the new database. Because ESEUTIL is creating a temp during the defrag and an actual copy when it is done, you must have at least 125% free space on the partition where your MDBDATA Information Stores directory is, or you will have to use a T/ switch and choose a temporary space on a network drive for the temp.edb file. Since the edb that you are defragging ends up getting deleted, the temp edb will need to go back over the network and this adds even more time to the disaster recovery.

Note: After the defrag is done, ESEUTIL considers the new edb to be a different edb from the original, and its log files cannot be replayed into the new database which is why the old log files within the MDBDATA were removed.

If you are fortunate enough to make it through these steps, the next Microsoft utility that you need is ISINTEG.

ISINTEG Information Store Integrity Check For Exchange Disaster Recovery

Now that ESEUTIL has done it’s thing you are ready to move on to ISINTEG. It is also located in the BIN folder, but it has different switches. The one you need for Exchange Disaster Recovery is:

ISINTEG -PRI -FIX -TEST ALLTESTS It might take a few times for this to come back without any errors, Microsoft actually warns not to run it more than 3 times, but I have heard people running it as many as 20 or 30! Needless to say they needed data recovery! Keep the ISINTEG to a minimum of 3 times just to be on the safe side.

You will have to repeat these steps on the PUB as well. It is OK to run all the ESEUTIL command on both the PUB and PRIV then run all of the ISINTEG afterwards, the important thing is to follow the order of /p, /d, BEFORE running ISINTEG!

While it is true that these utilities can deal with moderate corruption, they are not going to recover from any type of jet engine error. If you are having jet errors you will certainly need exchange server data recovery.

Another type of Exchange Recovery is here:

Before running any utilities COPY YOUR EDB FILE!

Data Recovery Steps: These commands can bring all edb’s to a consistent state.

• Syntax: ESEUTIL /r (3 character logfile base name)[options]
• Options: zero or more of the following switches, separated by a space:
• /l<path> - location of log files (default: current directory)
• /s<path> - location of system files (e.g., checkpoint file) (default: current directory)
• /i - ignore mismatched/missing database attachments
• /o - suppress logo

 Hopefully you have good backups and will never need to perform these types of Exchange Recovery.

Creation Of Clean priv1.edb File For Exchange Server IS

Many times during support calls the client is unable to restore from their Exchange Server backup. At that point if all the Microsoft Utilities like ESEUTIL or ISINTEG have failed, the client has no choice but to send their corrupted Exchange priv priv1 edb stm to a data recovery company.

How To Create A Blank Clean PRIV1 edb for Microsoft Exchange Server

The Exchange Information Stores are located in the MDBDATA folder (unless you have moved them to a custom location), usually on the second partition in a 2 partition Exchange Installation. In most cases the Information Stores Services are already shut down due to the tragedy that brought you to this page. If not, stop the Information Stores Services.

Now you will be able to rename the MDBDATA folder to something like MDBDATA-old or whatever you want. Now create a blank MDBDATA folder, go back and start your Information Stores Service.

At this point you can go into your Exchange Administration right click on the Private Mail Store (priv1.edb) and click “mount”. You will get a nag screen that complains about there being no edb and life will come to an end in the galaxy if your not careful, but ignore all that and hit OK. It now creates a brand new PRIV1.edb and PRIV1.stm file! In Exchange 5.5 there is only a PRIV.edb, but this works in version of Exchange, the only difference is that in 5.5 it pulls the user list out of the DIR.edb as opposed to the Active Directory in 200x.

That is all. Your users will have empty mailboxes, but at least they can send and receive mail until your Exchange data recovery company (hopefully us!) can return your mailboxes back. DTI Data Recovery is a full service hard drive recovery company that operates a class 100 clean room. If your Exchange Server has failed due to a hardware failure, please call 727-251-2058 for 24 hour support with Exchange Server Data Recovery.

***Disclaimer: After performing this process, it is extremely unlikely that Exchange Server Disaster Recovery will be possible with ESEUTIL! Copy your PRIV1 PUB1 edb and stm files before attempting any Exchange disaster recovery!

Exchange Server Dirty Shutdown Error

24 Hour Exchange Server Support

I run DTI Data’s 24 hour Microsoft Exchange Support Hotline. While I often jokingly refer to it as our “suicide hotline”, that is close to the truth. When that phone rings at 2 AM I know someone is having a serious Microsoft Exchange problem. I am often asked why we offer free phone support for Exchange. Most people expect there to be a hidden charge or some other hitch, in fact last night I got a call from the UK and the gentleman asked if the number was a premium line, one that costs money to call.

I explained to him like I explain to everyone that our business model is simple. In our experience (which is pretty vast) most Exchange errors are fatal. If you have had a dirty shutdown, jet engine errors or problems with transaction logs being corrupted or out of order, chances are you will have to restore from backup. ESEUTIL and ISINTEG do not really repair anything, they truncate parts of the database that appear to be in error, whether it is duplicate or illegal keys, page file header corruption, etc.

That is why we are so adamant about backing up the priv, priv1 edb and stm files before running any Microsoft Utilities!

So if most Exchange issues that are serious enough for a system engineer to call us are going to require a restoration of backup, how does that benefit us? Well unfortunately not all Exchange backup programs work all the time. After all Exchange is a database that is almost always active and there is always a chance that the backup will fail. That is why it is critical for users to check the backups at least monthly.

Now we get to the heart of the matter. Most engineers that call have had a serious Exchange issue, they have run utilities and that hasn’t worked, I’d say that at least 70% of the calls I get also have tried to restore from backup and it failed. SO if an admin is in need of Exchange Server Data Recovery, who are they going to send it to? Probably the company that answered the phone in the middle of the night and helped him get back up and running! That is why we offer free support, it’s a good business plan to spread good will and brand your company name as being customer oriented. DTI is very serious about customer service. Whether its a laptop hard drive recovery or a multiple hard disk RAID array, everyone is going to know that we are giving them the best we can.

The Most Common Exchange Server Failure Support Calls

Answering the emergency line for all these years has given me a lot of insight into common Exchange problems. While there certainly are plenty of things that can go wrong with an Exchange server, you would be surprised at how often I hear the same scenario of Exchange Server failure. Below are the most common calls that I receive from the support line.

Anti-Virus Deleted or Quarantined a Transaction Log

By far the most common call over the years has been the transaction log was quarantined by Norton, AVG or Panda. Lately this problem hasn’t been so apparent since there is so much good information about Exchange and anti-virus, but looking at the last 3 years this is by far the most predominant failure. The fact is you shouldn’t let any program have the ability to modify any component of Microsoft Exchange Server.

Information Store Has Reached The 16 GB Limit

Another historically rampant problem that has smoothed over recently is the priv or priv1 edb file size limitation. The Internet has also helped by educating admins about the 16 GB limit. Many didn’t know that the file size incorporated both the priv1 edb AND the priv1 stm file in its limitations. Once the size gets hit, the Information Stores can’t be mounted. There are plenty of websites (including our Exchange Forum) that posted the 1 GB temporary fix to the problem. This and Microsoft increasing the size to 72 GB has made this scenario less prevalent than say a year ago.

And now for the worst problem of them all…

Exchange Server Has Had A Dirty Shutdown

The big daddy problem of them all, a dirty shutdown is a very bad situation. This is the problem that I hear more than any other Exchange failure. A dirty shutdown happens whenever the Information Stores have not been closed in the normal manner. During the winter this is usually due to power surges or outages. Last week I took close to fifty calls from the Ohio area that had power problems leading to data recovery.

It is critical that your servers have UPS systems, battery backups, whatever you can do to help shut the Information Stores down properly during a power crises. If your server has had a dirty shutdown, exchange data recovery will be in your future if you don’t have a solid Exchange disaster recovery plan in place. A dirty shutdown is a disaster by definition. It often leads to jet engine errors, duplicate or illegal keys or worse. Keep your files backed up, and just as important: test your backups and recovery plan often!

Exchange Server Disaster Data Recovery

Microsoft Exchange Server Disaster Recovery

DTI Data is one of only a couple of real data recovery companies that also perform Exchange Server data recovery services. What separates us from other services is our programmers. All the data recovery software on this site was created right here in our labs. Add the network engineers to the mix and you get a team that can go down into the hex level of the Exchange database and fix duplicate keys or whatever problems are within the database pages.

I personally run the Exchange Server Data Recovery Emergency Hotline. I call it the suicide hotline since the majority of the callers are in very high stress situations. The top volume of problems revolve around Jet Engine Errors. These are serious problems that won’t be resolved with standard Exchange Server Disaster Recovery methods.

The most important fact that you need to take away from this post is:

BACKUP YOUR PRIV1 EDB’s and STM FILES BEFORE RUNNING ESEUTIL or ISINTEG!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

I can’t stress enough how important it is that you copy your Information Stores prior to running Microsoft utilities. Every single Exchange edb that we have received that was clean has been recovered. Every single one! Edb’s that have had utilities run on them can have data loss. You see ESEUTIL doesn’t actually “fix” anything. It truncates any records it feels are in error, that leads to data loss.

This series of posts will cover all the different problems that arise with Exchange Server Data Recovery.

Stay tuned for how to deal with a database that has reached the 16GB limit.