Scheduled maintenance - Saturday April 29 at 07:00 UTC Find out more     |     We have acquired Exinda More info

How to move, migrate, merge and split up Archive Stores

WARNING

It is recommended to create a backup of the system incl. the installation and data before going through this procedure.

This article outlines how to migrate items between Archive Stores. This can be useful in different situations, the most common ones are:
  • When there is the need to change the Archive Store type (for example from Firebird to Full-SQL or from Full-SQL to SQL+NTFS)
  • To reimport into a new set of Archive Stores if the existing Archive Store set does not align properly with the Archive Store naming or date range schema
  • If the database of an Archive Store was unrecoverably lost and only the binary folder is still available
  • To split large Archive Stores into multiple smaller ones
Depending on the GFI Archiver version used and the type of the source Archive Store the steps that need to be performed differ.

Before starting this procedure

  • Please read the important considerations marked in red below before starting with this procedure
  • It is recommended to update to the latest version before performing the migration
  • As items will be handled on disk it is recommended to disable anti-virus software completely during the whole process
  • Ensure that sufficient disk space is available on all disks involved incl. the disk on which the GFI Archiver installation is located
  • Reprocessing items in the way described will cause high load on system resources and might best be run outside of business hours


TO REIMPORT ITEMS VIA THE GFI ARCHIVER IMPORT SERVICE

STEP A: PLAN AND CONFIGURE THE DESIRED DESTINATION ARCHIVE STORES

  • GFI MailArchiver 2012 or newer
    1. Open the web interface and navigate to Configuration > Archive Stores
    2. Select New Archive Store Settings
    3. Follow the wizard and ensure that it matches the desired destination Archive Stores
    4. Select Schedule Archive
    5. Choose the desired time range (e.g. Monthly if 1x Archive Store should be created per month)

STEP B: VERIFY ADDALLOWNERSONIMPORT IS SET TO TRUE (GFI MAILARCHIVER 2012 OR NEWER ONLY)

  • IMPORTANT: It is vital to ensure that the configuration key AddAllOwnersOnImport in ..\Archiver\Core\Data\MArcSettings.xml is set to true.
  1. Stop all GFI Archiver services
  2. Browse to ...\GFI\Archiver\Core\Data
  3. Make a backup copy of MArcSettings.xml
    1. Create a subfolder named Backup
    2. Create a copy of the file MArcSettings.xml into the Backup subfolder
  4. Open MArcSettings.xml in a text editor
  5. Search for the key named AddAllOwnersOnImport
  6. Set its value to true
    • IMPORTANT: This value is case sensitive.
      • Example: AddAllOwnersOnImport="true"
  7. Save the changes
  8. Start all GFI Archiver services

STEP C: EXPORT ITEMS FROM SOURCE ARCHIVE STORE AND IMPORT TO DESTINATION ARCHIVE STORES

METHOD A (PREFERRED) - using the ExportSQLRawDataTool
  • Applies to: Full-SQL, SQL+NTFS, the database as well as the binary folder are still present and can be accessed
  • Does not apply to: Firebird

​Important considerations / preparations before starting this procedure:
  • The folder structure of emails is lost when migrating data via this procedure. However the emails can be re-synchronized to retrieve the folder structure, but only if they still exist in the user's Microsoft Exchange mailboxes. The folder structure of emails deleted from the Microsoft Exchange mailboxes is permanently lost.
  • Ownership between items and users which were deleted from Active Directory will be lost. After the re-import, no mailbox would exist within GFI Archiver anymore for such users. Affected items will be re-imported, but only be accessible via a user who has full access permissions.
    • For example: On 2012-12-01 UserA received an email and it was archived. UserA could access the email via the web interface and other methods within GFI Archiver. On 2013-06-01 UserA left the company and his user object in Active Directory was deleted. At this point a user with Full Access permissions within GFI Archiver could still access UserA's mailbox as ownership of emails was still present within the Archive Stores. On 2013-10-01 this procedure is used and items are reimported into new Archive Stores. At this point ownership between any items and UserA is lost. A user with Full Access permissions cannot add UserA's mailbox anymore. But he can browse and search against All User's items and ultimately access them.
    • If a mailbox is required you would need to create new user objects in Active Directory for users who were deleted in the past (one object per deleted user) and ensure that the new objects have set all email addresses that the original user had set in the past. This must be done before going through this procedure.
  • Ownership of items which was assigned based on distribution groups might be lost or might change (depending on changes of group membership).
    • For example: On 2012-12-01 UserA sent an email to a distribution group of which UserB and UserC were members at that time. When the email was originally archived ownership was assigned to all three users (UserA, UserB and UserC). On 2013-06-01 UserC was removed as a member from the distribution list and UserD was added as a member. On 2013-10-01 this procedure is used and items are reimported into new Archive Stores. At this point ownership will be assigned based on the current membership of the distribution list and ownership is assigned to UserA, UserB and UserD. UserC will not be assigned ownership. UserD will be assigned ownership although this user never received the email in the first place.
  • Ownership of items which was assigned based on archival through the GFI Archive Assistant will be lost.
    • For example: UserA has a PST file mounted in Outlook. It contained an email sent from UserB to UserC. The GFI Archive Assistant archived this email and ownership was assigned to UserA. After reimporting data via this procedure ownership of this email will not be assigned to UserA again, instead it will be assigned to UserB and UserC. Ownership for UserA would be lost.
 
  • It is recommended to update to the latest version before performing the migration
  • As items will be handled on disk it is recommended to disable anti-virus software completely during the whole process
  • Ensure that sufficient disk space is available on all disks involved incl. the disk on which the GFI Archiver installation is located
  • Reprocessing items in the way described will cause high load on system resources and might best be run outside of business hours
  1. Note down the Microsoft SQL server instance, the database name and the Path to binary source
    1. Open the web interface and navigate to Configuration > Archive Stores
    2. Highlight the source Archive Store > select Summary button 
  2. Remove the source Archive Store from the web interface
    1. Open the web interface and navigate to Configuration > Archive Stores
    2. Highlight the source Archive Store > select Detach button
  3. Start ExportSQLRawDataTool.exe
  4. Use the tool to connect to the desired Archive Store and export its items to a temp folder (for example to: d:\temp\2008)
  5. Start movebix.exe
    1. Open a command prompt (elevated) and navigate to the folder ...\GFI\Archiver\MAIS\bin
    2. Run the following command:
    • movebix.exe "<source binary path>" "<MAIS pickup folder>"
      • source binary path - the temp folder into which ExportSQLRawDataTool.exe saved the bix files
      • MAIS pickup folder - the folder which the Import service monitors for files. The default path is: ..\Archiver\MAIS\Pickup
    • For example: movebix.exe "d:\temp\2008" "c:\Program Files\GFI\Archiver\MAIS\Pickup"

METHOD B (FALLBACK) - if the ExportSQLRawDataTool cannot be used

​Important considerations which apply to this fallback method B in addition to the ones mentioned for method A:
  • Ownership information which was solely assigned based on the journal envelope (e.g. BCC recipients) will be lost.
    • For example: UserA sends an email to UserB. UserC is specified as a recipient via BCC. GFI Archiver archives this email and assigns ownership to all three users incl. the BCC recipient UserC (as the BCC recipient is available in the journal envelope). When reimporting this email via this procedure, the information that the email was sent to UserC via BCC is not present anymore and ownership will only get assigned to UserA and UserB.
  • Ownership of items which was assigned based on archival through the GFI Archive Assistant will be lost.​​
  • If you have archived any files using the GFI File Archive Assistant it is highly discouraged to follow this method. Any files which were archived using the GFI File Archive Assistant will lose the association with their owner and render them inaccessible for the user who archived them. Additionally, the item type will change from "file" to "email" during the process.

If the source Archive Store is Firebird or SQL+NTFS:
  1. Note the Path to binary source
    1. Open the web interface and navigate to Configuration > Archive Stores
    2. Highlight the source Archive Store > select Summary button 
  2. Remove the source Archive Store from the web interface
    1. Open the web interface and navigate to Configuration > Archive Stores
    2. Highlight the source Archive Store > select Detach button
  3. Make a backup copy of binary directory
  4. Start movebix.exe
    1. Open a command prompt (elevated) and navigate to the folder ...\GFI\Archiver\MAIS\bin
    2. Run the following command:
      • movebix.exe "<source binary path>" "<MAIS pickup folder>"
        • source binary path - the binary folder of the source Archive Store (which was noted down in a previous step)
        • MAIS pickup folder - the folder which the Import service monitors for files. The default path is: ..\Archiver\MAIS\Pickup
      • For example: movebix.exe "c:\Program Files\GFI\Archiver\Databases\2008\binary" "c:\Program Files\GFI\Archiver\MAIS\Pickup"
    • NOTE: Items can be copied instead of moved via copybix.exe. Movebix.exe is recommended though as it preserves disk space and makes the process more trackable.
If the source Archive Store is Full-SQL:
  1. Export items
    1. Navigate to Start > Programs > GFI Archiver > GFI Archiver Import Export Tool
    2. Choose the mode Download emails from GFI Archiver
    3. Connect to GFI Archiver URL as a member of the Full Access Group
    4. Choose the option What do you want to export? Items with specific search criteria
    5. Export all items of the source Archive Store
      • NOTES:
      • As this export relies on Search it is vital that the index of the source Archive Store is healthy. If the search does not return the desired results you might need to rebuild the index.
      • Use the option Restrict search on particular archive store
      • Untick the option Retrieve only the first 5000 occurrences
      • Remove any Query Criteria lines to ensure all items are returned
    6. From the Export and Download Options page, select
      • Download to path
      • Download emails in EML format
    7. Select Next to start the export process
    8. When the export process completes, select Finish 
      • NOTE: The duration of the export process depends on the amount of items to export.
  2. Remove the source Archive Store from web interface
    1. Open the web interface and navigate to Configuration > Archive Stores
    2. Highlight the source Archive Store > select Detach button
  3. Rename .EML files to .Q
    1. Browse to the directory that was specified when downloading the items 
    2. In this location, there will be a folders named Folder_# for every 5000 items that were exported.
      1. For example, if 7000 items were exported, Folder_1 would contain the first 5000 items and Folder_2 would contain the remaining 2000 items.
    3. Open a command prompt and change to the Folder_1 directory
    4. Execute the command
      • ren *.eml *.q
    5. Repeat the steps above for each of the Folder_# directories
  4. Move the *.q files from the Folder_# directories to ...\GFI\Archiver\MAIS\Pickup in batches
    • NOTE: It is recommended to move the files from the Folder_# directories one at a time and only continue with the next one once ..\Archiver\MAIS\Pickup is cleared.



TO SPLIT A LARGE MICROSOFT SQL BASED ARCHIVE STORE INTO MULTIPLE SMALLER ARCHIVE STORES

Use the DBSplitTool if you have a very large Microsoft SQL based Archive Store (Full-SQL or SQL+NTFS) and need to split it into multiple smaller ones and do not have any other requirements.

This tools uses a different approach compared to the extended procedures above. It connects to the Mircrosoft SQL database (and binary folder) and directly copies the data into new Archive Stores based on a a chosen schedule.

Advantages

  • The tool does carry over the folder structure and ownership information.

Disadvantages

  • The tool does not store data in existing Archive Stores, even if any are present which cover the same date range of the items which are to be processed.