shadow server api
In this blog, we will be using the api provided by the shadow server and obtain information. We will use the official API to pull data from the Shadow Server systems. Previously this has been delivered manually via email but it be manual process where we can configure the intelmq to fetch the email from the email server obtain the attachment and delete the file. But from my perspective, this seems ineffective and not the proper way to do it.
Use the below command to clone the official repo.
git clone https://github.com/The-Shadowserver-Foundation/api_utils.git
As provided in the guide, if we want to use the API we will have to enter the following information in the
[api] key = <<API-KEY>> secret = <<SECRET>> uri = https://transform.shadowserver.org/api2/
When I try to run it, I get the following message
reza@tools:~/Desktop/shadowserver/api_utils$ python3 report-manager.py Usage: report_manager.py /path/to/config.ini
After all the information has been entered into the appropriate locations, we can now run the script so that we can download the CSV file containing report for last two days.
reza@tools:~/Desktop/shadowserver/api_utils$ python3 report-manager.py report-manager-config.ini
This command might take a while to execute. After this commands successful execution, we can see a lot of file downloaded into the path specifies in the
Some sample configuration has been given below:
[reports] directory = /var/tmp/reports min_disk_free = 512 notifier = none url_prefix = http://myserver/reports/ [stomp] server = 127.0.0.1 port = 61613 user = guest password = guest queue = /queue/mytest [redis] server = 127.0.0.1 port = 6379 ;password = guest queue = mytest [kafka] server = 127.0.0.1 port = 9092 queue = mytest
The above configuration is for all the every
notifier how ever we will be only using
redis in our current environment. So we will change it to:
[reports] directory = /var/tmp/reports min_disk_free = 512 notifier = redis url_prefix = http://myserver/reports/ [redis] server = 127.0.0.1 port = 6379 ;password = guest queue = ss-report-manager
Here you can see that change has been performed in the notifier’s value. Now lets re-execute the with updated configuration.
$ python3 report-manager.py report-manager-config.ini
After a while we should be able to see some data in
db0 under the queue
The downloaded files are now stored in
/var/tmp/reports. Basically this path is considered as the root path for the web server. From here, you can build a webserver and instruct other systems to pull data from here.
Lets jump back to the IntelMQ system and start all the bots