WebSpy is a Fastvue Product

Support Center

Data Storage Guide and Best Practices

Last Updated: Mar 20, 2018 06:13PM PDT
Trying to balance data storage requirements with available disk space resources can be a challenge. Here are some ways organizations are using WebSpy Vantage to help.

1. Store as much of the original log data as disk space allows.

Some organizations have compliance requirements to store x amount of historical log data. This data is transportable, readable, and non-proprietary. If you rely on your WebSpy Storage as a long-term data storage, you'll always need WebSpy Vantage to analyze it (great for us, but not a great log data storage strategy).

If you're using our free Fastvue Syslog tool to capture syslog data into raw text files, this application zips and moves files older than a specified age (default is 30 days) to an archive folder. You can then write a script to delete old archive files older than your maximum retention policy.

Disk space is usually not expensive, and you may want to consider services like Amazon Glacier for cheap cloud-based long-term storage of archived log data.

2. Keep enough log data in a WebSpy Vantage storage to cover short-term ad-hoc reporting requests.

We find that if someone comes to you asking for specific reports, this is generally for the past week or maybe the past month.

You can always go and import the original logs for the period requested into a separate storage, run your report, then delete the storage, but the import process could be more time-consuming than you'd like, especially for an urgent request.

We find keeping 3 months of WebSpy Storage data at hand if your disk space allows is usually more than enough. This is controlled using the Purge data from storage task action.

You can also employ Data Deduplication to minimize the data size on disk of your WebSpy Storage.

3. Publish Reports to the Web Module for long-term information storage

The report files themselves are much smaller than the original log data, so you can keep many more of these - say 12 months and beyond.

You can then use the Dynamic Reports tab to collate the daily reports into weekly or monthly reports (or any timeframe), so this is a great way to provide access to information over the long term without eating too much hard drive space.

As always, if you have any specific questions around data storage and reporting, please get in touch!

Contact Us

seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
Invalid characters found