Return to July 2009 SPI Connections

Filtering Files Based on File Size

Synchronizing files over a slow connection or a connection where the synchronization job must be mindful of consuming excessive bandwidth can be a challenge for a network administrator. This challenge can be amplified to even greater levels when the administrator must synchronize large files on a regular basis. SureSync’s synchronization engine supports the ability to filter files based on the size of the files being processed. This functionality can be helpful in a number of different situations. Consider the following scenarios:

• Your network consists of multiple remote offices connected by slow data lines such as DSL lines. Bandwidth is limited during the day and must be made available for end user uses as much as possible during the day. You would like to synchronize only small files during the day.
• Your environment features a large quantity of big files. These files are changing regularly and this causes delays in the processing of small files because those small files end up stuck in the queue behind the large files. You would like to speed up processing of the small files.

Our Example

In our example environment, we have a need to synchronize a folder named SalesData between two machines. This folder contains a number of large files and a number of small files. We want to break the data up so that small files are processed before large files. We want to do this because the two offices are connected by a slow connection.

Filtering on File Size with a Schedule

Our first scenario will involve performing this synchronization with a Schedule. You can configure the Schedule to run at any frequency you like. However, we will be running this synchronization job once per day starting at 4PM. We will copy files smaller than 10MB first and then files larger than 10MB second.

First, create your Relation. Define your root paths and all other options as you work through the wizards. We will use the name SalesData as the name of the Relation in this example.

Second, create a Schedule. Name the Schedule something like “SalesData – Small Files”. Configure the options for when you want the Schedule to run like you normally would. When you’re done with the wizard, click on the Options tab. For ‘Process file sizes’ select ‘smaller than’ and then enter in 10,240KB (which is 10MB). See screenshot below.

Create a second Schedule and name this something like “SalesData – Large Files”. Place this Schedule on hold so it doesn’t run itself. We will be triggering this Schedule to run after the first Schedule completes using the SyncCmd command line utility.

On the options tab for the “SalesData – Large Files” Schedule, select ‘equal to or larger than’ for the ‘Process file sizes’ option and enter 10,240KB.

Now click on the “SalesData – Small Files” Schedule and then the ‘Scripts’ tab.

We will enter a SyncCmd command under ‘Script for end of each Schedule’ which will run the “SalesData – Large Files” Schedule automatically when this Schedule completes.

The command would be (including all quotes):

"C:\Program Files\Software Pursuits\SureSync\SyncCmd.exe" /XStart /S"SalesData - Large Files"

Also, set the ‘Run ending script’ option to ‘1-Always’. This ensures that the second Schedule is always kicked off, even if an error is encountered in the first Schedule.

The end result of this configuration is that the SalesData Relation will run on a Scheduled basis each day at 4PM by the “SalesData – Small Files” Schedule. The Schedule will transfer all files smaller than 10MB and then will kick off the “SalesData – Large Files” Schedule automatically to transfer all files larger than 10MB.

Filtering on File Size with a Real-Time Monitor

With the Real-Time Monitor, we are going to approach things a little differently. Real-Time Monitors have a feature called Intervals. An Interval allows you to configure the monitor to run a single Relation under different priorities during different spans of time. You can also configure the Intervals to apply to different size files.

For the Real-Time Monitor example we will configure the Monitor to copy files smaller than 10MB from 7AM to 6PM and then transfer files larger than 10MB from 6PM to 7AM each day.

First, create your Relation. Define your root paths and all other options as you work through the wizards. We will use the name SalesData as the name of the Relation in this example.
Second, create your Real-Time Monitor. For this example, we will call the Real-Time Monitor “SalesData RTM”.

Once you’re done with the Real-Time Monitor wizard, click on the Real-Time Monitor and then go to the Intervals tab. Select the existing Interval and click Edit.

Change the ‘Interval Start Time’ option to 07:00AM and the ‘Interval End Time’ to 06:00PM. Then set the ‘Process only files with file sizes’ option to ‘smaller than’ and enter 10,240KB. Then click ‘Save’.

Then click ‘Add’ to create a second Interval.

Change the ‘Interval Start Time’ option to 06:00PM and the ‘Interval End Time’ to 07:00AM. Then set the ‘Process only files with file sizes’ option to ‘equal to or larger than’ and enter 10,240KB. Then click ‘Save’.

The end result of this configuration is that “SalesData” will run on a real-time basis. As files change during the day, if they are smaller than 10MB they will be transferred. If the files are larger than 10MB they will be queued to be transferred at the nighttime interval. This allows small files to be transferred during the workday and keeps the large files from consuming too much bandwidth or slowing down the transmission of small files.

Conclusion

Configuring your synchronizations to filter based on file size is a useful way to address bandwidth and availability issues when dealing with low bandwidth and slow connection environments. If you need assistance implementing file size filtering in your SureSync environment, please contact our Support Team for further assistance.

Return to July 2009 SPI Connections

Software Pursuits, Inc.
1500 Fashion Island Blvd.
San Mateo, CA 94404

You are receiving the Software Pursuits Connection because you are a Software Pursuits' customer or you indicated that you would like to be updated on Software Pursuits' products. The Software Pursuits Connection keeps you connected to Software Pursuits' product releases, upgrades, product tips, and solutions our products will help you implement. If you have any questions or comments, please send an email to the Editor.

If you would like to Unsubscribe, please follow this link.

(c) Software Pursuits, Inc., 2009. SureSync is a registered trademark. The product and service offerings described in this document are produced by Software Pursuits, Inc. This document includes many forward-looking statements regarding anticipated product functionality, targeted release dates, availability of technology and certain company initiatives that involve risks and uncertainties, and actual results may differ materially. This document is not intended to obligate Software Pursuits to such functionality release dates, technology, initiatives and plans, but merely to inform the audience of Software Pursuits current intentions. This document is for informational purposes only and is subject to change without notice. Software Pursuits makes no warranties, express or implied in this document. The contents of this document are believed to be current and accurate as of its date of publication. For a complete description of the product features, please refer to the product's user guides, reference manuals and release notes. This newsletter was sent to you because you are a Software Pursuits customer.