WebSpy is a Fastvue Product

Support Center

Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Webspy data source

Mike Nov 02, 2016 12:12PM PDT

We have an implementation of Splunk running for our enterprise. Can Webspy connect to the Splunk database and use it as a data source for reporting? All of our Palo Alto logs are sent to Splunk already.

Thanks
Mike

Up 0 rated Down
Fastvue Nov 02, 2016 06:22PM PDT FASTVUE Agent
Hey Mike,

Thanks for getting in touch about this. Unfortunately we can't connect to Splunk's database directly, but you can configure Splunk to forward syslog data to another syslog server that logs to text files that can be imported into WebSpy Vantage. http://docs.splunk.com/Documentation/Splunk/6.0/Forwarding/Forwarddatatothird-partysystemsd#Syslog_data

You can use a syslog server of your choice for this such as Kiwi Syslog, however we've recently developed our own syslog server that is completely free and unlimited for Windows. You can check it out and download it here: http://www.fastvue.co/syslog

Alternatively, you can configure an additional syslog server in Palo Alto directly instead of forwarding from Splunk.

I hope this helps! Please let us know how you go.

Cheers!
Scott
Up 0 rated Down
Mike Ramotar Nov 08, 2016 04:49PM PST
Thanks Scott, that was very helpful!

Would it be recommended to setup a single Windows server with your syslog solution and WebSpy together? Does that mean that WebSpy will read from that syslog server directly? Or does it have to be exported from that syslog server into WebSpy?

I am just thinking about space on the syslog server. Once it is imported into WebSpy, do we still need that data in the syslog server? Or can it be purged?

Thanks in advance for your assistance.

Mike
Up 0 rated Down
Fastvue Nov 08, 2016 05:20PM PST FASTVUE Agent
Hey Mike,

The Fastvue Syslog Server will create text files in the folder that you specify, and then zip and archive files once they reach a certain age. The default age for archiving is 30 days, but if you need to conserve disk space, you can move that to something sooner such as 7 days. The syslog server does not delete the zipped archive files at this stage, so that would be a manual process. But text logs do compress quite well.

In WebSpy Vantage, you'd setup a Storage to read from the syslog server's log folder (not the archive folder), and set a schedule to import logs each day. WebSpy Vantage does 'duplicate' the log data in it's Storage, and you can configure a data retention policy on the storage by adding a 'Purge data from storage' action to your daily task, and purge data older than 7 days for example.

It's important to note that once data is imported into a Storage, there is no physical link back to the original log files. So you can archive or delete the original syslog files, and just keep data in your WebSpy Vantage storage. But... be aware that the WebSpy Storage is a binary proprietary format, and can only be read by running a report in WebSpy Vantage. Text logs are open, human readable and all that good stuff, so we recommend archiving and keeping the original text logs for as long as possible, and keep the data in your WebSpy Vantage storage to a minimum.

I'd recommend running your reports as part of your daily schedule, and once you have the reports you can purge the data from your storage (A report is static document, so there's no problem deleting data from the storage once the report is generated). If you ever need to report on older data again, you can simply import the old archived (zipped) logs from the syslog server's archive into a separate storage, run your reports, then delete the storage.

You can also use the Dynamic Reports tab in the web module to collate your daily reports together to generate weekly, monthly or yearly reports.

So to summarise:
1. Keep text logs (including archives) from the syslog server for as long as possible, and set the archive age to a short time frame if disk space is an issue.
2. Import your syslog logs into Vantage on a daily schedule, and purge data older than say 7 days from the storage on the schedule as well.
3. Publish reports to your Web Module each day as part of your schedule (make sure you add a relative date filter to the report task so that it only reports on the past 1 day)
4. Use the Dynamic Reports tab to collate daily reports together. This way you don't need older data in your storage to report across months.
5. If needed, manually delete the Archive files from the syslog server when they reach a certain age - such as 1 year.

I hope that makes sense! Let us know if you have any further questions.

Cheers!
Scott

Post Your Public Answer

Your name (required)
Your email address (required)
Answer (required)

Contact Us

support@fastvue.co
http://assets0.desk.com/
false
desk
Loading
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
about
false
Invalid characters found
/customer/en/portal/articles/autocomplete