Cuckoo 2.0. Putting together the best open source malware analysis platform

  • Tutorial

Cuckoo Sandbox logo
Greetings Habr!


4 years ago I published the assembly instructions for the Cuckoo Sandbox 1.2 dynamic malware analysis platform. During this time, the project managed to gain impressive functionality and a huge community, recently updated to version 2.0, which has been hanging in the Release Candidate stage for more than a year and a half.


I finally prepared for you the complete Cuckoo Sandbox 2.0.5 assembly manual with all the goodies that are currently in the project, with Cuckoo packaging in venv and without root , adding more than 12 security utilities to the sandbox! Based on the aggregate of the material collected, I’m not afraid to say: "This is the most complete and verified step by step guide of all published on the Internet at the moment." For those who master the article to the end - I will give a little advice on what to do when the sandbox is assembled and how to get the maximum profit from the automation of information security processes with your own siil using open source. To all geeks, virus analysts, security guards, guys from SOC, CERT, CSIRT and just lovers to poke buttons in the terminal - welcome to kat!


A lot has happened in the Cuckoo project over the past few years, new tools have been added, support for additional software has appeared, the number of signatures has increased significantly, a malware rating system for the scanned file has appeared, the entire UI has been rewritten and now you can set up a nice dark theme.


black UI


Positive changes in the project are obvious - the sandbox began to be updated with micro releases, which quickly close identified and confirmed bugs. The developers began to write about all the small changes in the blog , and there you can find the roadmap of the project. Unfortunately, there have been so many changes that it’s difficult to compile a “before” and “after” summary table, now these are 2 platforms of different eras, you better watch the blog yourself , you will like it!


Release cycle


I advise you to check the blog more often, the developers of which release promise to include the zer0mon VM anti-detection driver in the master branch , but now the deadlines will move to release 2.1.0, stay tuned, the thing is really necessary!

The Cuckoo code itself moved to the PIP package manager and the cuckoo is updated literally in one command.


Positive was the news about the launch of commercial support for customers who need stability in their work, since stability is the main drawback of Cuckoo from the first releases. I really hope that the money raised in the project will help developers to achieve high-quality operation of the application.


Cuckoo Enterpise


However, the installation problem has not changed since the start of the project - Cuckoo Sandbox is well documented, and with modules that always have bugs, this is a disaster. Some applications are not documented at all, and to understand how and what works - you need to look at the code, some are almost not updated and are not supported, like HoneyD. Having spent a considerable amount of time, I was able to collect the cuckoo with all the recommendations of the developers - wrap the cuckoo in Venv and configure everything from scratch without using root privileges for Cuckoo to work! Each step is verified step by step by multiple rollbacks of backups in a test environment until the instruction has become fully operational.


Build Features


According to the recommendations of the developers, Cuckoo Sandbox 2.0.5 is built on the Ubuntu OS 04/16/03 installed in the ESXi cluster. As the Cuckoo hypervisor, VirtualBox 5.2 will be used. VM installation will be performed through the VMcloak auto- installation script . With VMcloak, you no longer need to look for vulnerable old software for the guest OS, you do not need to configure the network and Python, VMcloak installs everything you need, including the Cuckoo Agent, and sets it up in one terminal command. Cloning, administering and tuning VMs is now a pleasure with VMcloak. I use Windows 7 SP1 X64 as the virtual environment OS, although Cuckoo Sandbox supports all versions of Windows starting with XP.
Half a year ago, Cuckoo developers included the ability to proxy traffic from a VM through a VPN or Tor. In this setting, I settled on a simple method - to give access to the Internet directly or through Tor.


In addition to Cuckoo Sandbox, the following software will be installed:


  1. Suricata - IDS.
  2. Snort - IDS.
  3. Honeyd - Honeypot.
  4. InetSim - Network emulator.
  5. Tor - onion routing of guest VM traffic.
  6. Teserract - Recognize text in screenshots.
  7. Mitmproxy - Analysis of SSL traffic through "man-in-the-middle".
  8. Moloch - IDS (a very convenient and cool utility for traffic analysis).
  9. SSDeep - Fuzzy-Hashing.
  10. Volatility - RAM exploration framework.
  11. Distorm3 - Disassembler.
  12. Yara - A utility for recognizing and classifying malware.

The thirteenth point was supposed to be a program for streaming anti-virus analysis of IRMA files , however, when IRMA was included in Cuckoo, the report uploading in Elastic broke down, the developers still did not give an answer for this bug, and there was no time to analyze the Elastic Index. Therefore, in this manual there will be an item about building IRMA, but integration with Cuckoo will not be included. Perhaps there is a guru of the ELK stack, ready to bravely overcome the bug and complement my article. You can find and use IRMA on port 8080.


As a drawback, I can point out incorrect links to Moloch in the web interface of the sandbox, you can probably fix it, but for me this is not critical, the Moloch service itself will hang on port 8005 and this is enough.


moloch bug


Cuckoo will store skin data in MongoDB, system data in Postgresql and indexes in Elasticsearch for quick search and selection, export to Json, HTML and PDF, work through the Nginx web server with SSL configured across all canons (not counting self-signed cert) and in conjunction with UWSGI.


This assembly is configured only for analysis in the Windows sandbox. Android, OS X, and Linux are not configured. If you have questions on this part - write to the PM, theoretically I know how to do this, but this functionality is beyond the scope of my interests and my work at the moment.


Manual Features


In the current article, the auto-install script will not be provided. Of course, I’m not too lazy to write it, but it doesn’t make sense for the sandbox to work correctly - it’s better to collect it manually and carefully look at what and how it is going to analyze each error. You still collect an application that interacts with a dozen external programs that are constantly updated. Bugs are inevitable, so it's better to find them yourself than to figure out later what went wrong in the script.


In this article I did not begin to take all the dependencies of the installed programs to the very beginning with one line, as in the previous article. For each specific software connected to Cuckoo, the dependencies are indicated in the installation section of this software.


4 years of malware analysis using Cuckoo showed me an obvious linear relationship - the more you connect to the sandbox of services, the worse it works. Therefore, it makes no sense to put all the dependencies at once, because older IT experts can give up on the market those components that work unstably, and the information from them will be redundant. The best is the enemy of the good, as they say.


The REST API is not involved in this article , it is quite simple to do this, but I do not need it at the current stage.


I won’t describe each utility in detail, I’ll only mark the important, the more significant part of the utilities is described in my last article , there is also infa on enabling the REST API.


And most importantly - the article is published for several months after the assembly instructions, something may already change, something will not work. If you meet a bug - write to the PM.


Now we will answer the most popular question and proceed with the installation!


But what if I'm not good at your Linux, and the sandbox is really needed


The answer to this question is simple - use online sandboxes or mess with Windows Sysinternals .


  • Service number 1, "Estonian Cuckoo" . Very competently assembled Cuckoo Sandbox latest version from Estonian CERT (Computer Emergency Response Team). The best sandbox implementation I've ever found on the net. Using the service, remember that the downloaded data can be disassembled into competency indicators (IoC) and, using software like MISP, transferred to other national CERTs in accordance with established TLP transfer rules . Therefore, carefully consider whether or not to upload confidential software to you.
  • Service number 2, "Hybrid Analysis" . A popular sandbox in narrow circles (or no longer in narrow ones) is an online sandbox, the stability and quality of which is on top, of the minuses - the functionality is a little limited, malware samples cannot be downloaded.
  • Service number 3, "Ever-Dead Malwr" - the most popular sandbox supported by the authors of Cuckoo Sandbox, has now fallen under load again and has not been rising for half a year. The only site from where it was possible to load samples of virus software for analysis for free, not counting this collection . I hope for a revival, but the forecast is not yet encouraging.
  • Option 4, “I’ll look for myself” is for those who are familiar with OSINT principles and the language of search queries. We take a piece of static text from the platform:
    And google in quotes, finding all the sandboxes that are in the index except github:
    google osint

Assembly


So, the theoretical part is behind, we begin to practice!


OS update


We update system packages.


sudo apt update
sudo apt upgrade -y

Cuckoo Dependencies


Install the Cuckoo dependencies.


sudo apt install python python-pip python-dev libffi-dev libssl-dev libfuzzy-dev -y
sudo apt install python-virtualenv python-setuptools -y
sudo apt install libjpeg-dev zlib1g-dev swig -y
sudo -H pip install -U pip

Virtualbox


Install Virtualbox with extpack.


cd /opt
sudo sh -c 'echo "deb http://download.virtualbox.org/virtualbox/debian xenial contrib" >> /etc/apt/sources.list.d/virtualbox.list'
sudo wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
sudo apt update && sudo apt install virtualbox-5.2 -y
VBOX_LATEST_VERSION=$(curl http://download.virtualbox.org/virtualbox/LATEST.TXT)
sudo wget http://download.virtualbox.org/virtualbox/${VBOX_LATEST_VERSION}/Oracle_VM_VirtualBox_Extension_Pack-${VBOX_LATEST_VERSION}.vbox-extpack
sudo vboxmanage extpack install Oracle_VM_VirtualBox_Extension_Pack-${VBOX_LATEST_VERSION}.vbox-extpack

Cuckoo sandbox


Install Venv, activate and install Cuckoo via PIP.


sudo adduser cuckoo
sudo usermod -a -G vboxusers cuckoo
cd /home/cuckoo
su cuckoo
virtualenv cuckoo
. /home/cuckoo/cuckoo/bin/activate
pip install -U pip setuptools psycopg2 yara-python weasyprint pycrypto pydeep
easy_install distribute
pip install -U cuckoo
pip install weasyprint==0.36
pip install m2crypto==0.24.0
cuckoo
cuckoo community
deactivate
sudo apt install python-m2crypto
exit

Java for Elasticsearch


Elastic is written in Java, so we need to install Java.


sudo add-apt-repository ppa:webupd8team/java
sudo apt update && sudo apt install oracle-java8-installer -y
sudo bash -c "echo 'JAVA_HOME=\"/usr/lib/jvm/java-8-openjdk-amd64\"' >> /etc/environment"
source /etc/environment

Db


Here we install and configure all the databases, note that the variable "db_passwd" generates a random password for the Postgres database, if you want to set your own, do not forget to set it.
Cuckoo uses the obsolete Elastic 2nd version, pay attention to this and do not accidentally install the 5.x + version.


sudo apt install mongodb -y
sudo apt install postgresql libpq-dev -y
sudo pip install psycopg2
db_passwd=$(date +%s | sha256sum | base64 | head -c 32 ; echo)
echo "CREATE USER cuckoo WITH PASSWORD '$db_passwd';" | sudo -u postgres psql
echo "CREATE DATABASE cuckoo;" | sudo -u postgres psql
echo "GRANT ALL PRIVILEGES ON DATABASE cuckoo to cuckoo;" | sudo -u postgres psql
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list
sudo apt update && sudo apt install elasticsearch -y
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
sudo service elasticsearch stop
cd /home/cuckoo/
sudo mkdir /home/cuckoo/ESData
sudo chown root:elasticsearch ESData
sudo chmod 777 /home/cuckoo/ESData
sudo usermod -a -G elasticsearch cuckoo
sudo bash -c "cat >> /etc/elasticsearch/elasticsearch.yml <<DELIM
cluster.name: es-cuckoo
node.name: es-node-n1
node.master: true
node.data: true
bootstrap.mlockall: true
path.data: /home/cuckoo/ESData
network.bind_host: 0.0.0.0
DELIM"
sudo service elasticsearch start
sudo curl -X PUT -d @'/home/cuckoo/.cuckoo/elasticsearch/template.json' 'http://localhost:9200/_template/cuckoo'

Yara + rules


Install the latest version of Yara and add the Yara rules to Cuckoo.


cd /opt
sudo apt install dh-autoreconf flex bison libjansson-dev libmagic-dev -y
sudo wget https://github.com/VirusTotal/yara/archive/v3.7.1.tar.gz
sudo tar -zxf v3.7.1.tar.gz
cd yara-3.6.3/
sudo ./bootstrap.sh
sudo ./configure --with-crypto --enable-cuckoo --enable-magic
sudo make
sudo make install
sudo -H pip install -U yara-python
cd /home/cuckoo/.cuckoo/yara/
su cuckoo
sudo git clone https://github.com/lehuff/cuckoo-yara-rules.git
sudo cp cuckoo-yara-rules/cuckoo-yara-rules.py .
sudo rm -rf cuckoo-yara-rules
sudo python cuckoo-yara-rules.py
sudo chown -R cuckoo:cuckoo /home/cuckoo/.cuckoo/

SSDeep


Install SSDeep.


cd /opt
sudo -H pip install -U ssdeep
sudo git clone https://github.com/bunzen/pySSDeep.git
cd pySSDeep
sudo python setup.py build
sudo python setup.py install
cd -

Volatility


Installing Volatility is simple, but getting Cuckoo from venv to see it is not very good, installing it in venv along with Cuckoo is also not an option, it changes the versions of Cuckoo libraries with dependencies. On the third line, a crutch that solves this problem, if anyone has an idea how to make friends in a more faithful way - write.


sudo apt install pcregrep libpcre++-dev -y
sudo -H pip install -U git+https://github.com/kbandla/pydeep.git
sudo apt install volatility -y
cp -r /usr/lib/python2.7/dist-packages/volatility* /home/cuckoo/cuckoo/lib/python2.7/site-packages
sudo chown cuckoo:cuckoo /home/cuckoo/cuckoo/lib/python2.7/site-packages/*
mv /home/cuckoo/.cuckoo/signatures/windows/volatility_sig.py /home/cuckoo/.cuckoo/signatures/windows/volatility_sig.py.deactivate

TCPDump


Reached TCP dump.


sudo apt install tcpdump apparmor-utils -y
sudo aa-disable /usr/sbin/tcpdump
sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump
sudo chmod +s /usr/sbin/tcpdump

Teserract


It is set simply, it connects too, but I did not notice the charms in the work of OCR.


sudo apt install tesseract-ocr -y

Fonts for pdf


Without this magic PDF reports will not be generated.


sudo -H pip install -U cairocffi
sudo apt install wkhtmltopdf xvfb xfonts-100dpi -y

Mitmproxy


A very useful library that allows you to snoop on SSL traffic. Please note Cuckoo only understands version 0.18.2 package.


sudo apt install libarchive13 libxml2-dev libxslt1-dev -y
sudo -H pip install -U mitmproxy==0.18.2
su cuckoo
cd ~
mitmproxy + ctrl-c
sudo cp ~/.mitmproxy/mitmproxy-ca-cert.p12 /home/cuckoo/.cuckoo/analyzer/windows/bin/cert.p12
sudo chown cuckoo:cuckoo /home/cuckoo/.cuckoo/analyzer/windows/bin/cert.p12
exit

Tor


Everything is quite simple here.


sudo apt install tor -y
sudo sh -c 'echo TransPort 192.168.56.1:9040 >> /etc/tor/torrc'
sudo sh -c 'echo DNSPort 192.168.56.1:5353 >> /etc/tor/torrc'

Suricata


Initially, I wanted to write what and where in the config I need to edit, but by the time I was able to overcome Suricata and remove all the flaws of work, a lot of edits were made to the config, and 0 was documented, so I publish the whole config. Moreover, the original rules file will remain untouched.


sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt update && sudo apt install suricata -y
echo "alert http any any -> any any (msg:\"FILE store all\"; filestore; noalert; sid:15; rev:1;)"  | sudo tee /etc/suricata/rules/cuckoo.rules
sudo touch /etc/suricata/suricata-cuckoo.yaml

/etc/suricata/suricata-cuckoo.yaml
%YAML 1.1
---
vars:
  address-groups:
    HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
    EXTERNAL_NET: "any"
    HTTP_SERVERS: "$HOME_NET"
    SMTP_SERVERS: "$HOME_NET"
    SQL_SERVERS: "$HOME_NET"
    DNS_SERVERS: "$HOME_NET"
    TELNET_SERVERS: "$HOME_NET"
    AIM_SERVERS: "$EXTERNAL_NET"
    DNP3_SERVER: "$HOME_NET"
    DNP3_CLIENT: "$HOME_NET"
    MODBUS_CLIENT: "$HOME_NET"
    MODBUS_SERVER: "$HOME_NET"
    ENIP_CLIENT: "$HOME_NET"
    ENIP_SERVER: "$HOME_NET"
  port-groups:
    HTTP_PORTS: "80"
    SHELLCODE_PORTS: "!80"
    ORACLE_PORTS: 1521
    SSH_PORTS: 22
    DNP3_PORTS: 20000
    MODBUS_PORTS: 502
    FILE_DATA_PORTS: "[$HTTP_PORTS,110,143]"
    FTP_PORTS: 21
default-rule-path: /etc/suricata/rules
rule-files:
 - botcc.rules
 - ciarmy.rules
 - compromised.rules
 - drop.rules
 - dshield.rules
 - emerging-attack_response.rules
 - emerging-chat.rules
 - emerging-current_events.rules
 - emerging-dns.rules
 - emerging-dos.rules
 - emerging-exploit.rules
 - emerging-ftp.rules
 - emerging-imap.rules
 - emerging-malware.rules
 - emerging-misc.rules
 - emerging-mobile_malware.rules
 - emerging-netbios.rules
 - emerging-p2p.rules
 - emerging-policy.rules
 - emerging-pop3.rules
 - emerging-rpc.rules
 - emerging-scan.rules
 - emerging-smtp.rules
 - emerging-snmp.rules
 - emerging-sql.rules
 - emerging-telnet.rules
 - emerging-tftp.rules
 - emerging-trojan.rules
 - emerging-user_agents.rules
 - emerging-voip.rules
 - emerging-web_client.rules
 - emerging-web_server.rules
 - emerging-worm.rules
 - tor.rules
 - http-events.rules
 - smtp-events.rules
 - dns-events.rules
 - tls-events.rules
classification-file: /etc/suricata/classification.config
reference-config-file: /etc/suricata/reference.config
default-log-dir: /var/log/suricata/
stats:
  enabled: yes
  interval: 8
outputs:
  - fast:
      enabled: no
      filename: fast.log
      append: yes
  - eve-log:
      enabled: yes
      filetype: regular
      filename: eve.json
      types:
        - alert:
            metadata: yes
            tagged-packets: yes
            xff:
              enabled: no
              mode: extra-data
              deployment: reverse
              header: X-Forwarded-For
        - http:
            extended: yes
        - dns:
            query: yes
            answer: yes
        - tls:
            extended: yes
        - files:
            force-magic: no
        - smtp:
        - ssh
        - stats:
            totals: yes
            threads: no
            deltas: no=
        - flow
  - unified2-alert:
      enabled: no
      filename: unified2.alert
      xff:
        enabled: no
        mode: extra-data
        deployment: reverse
        header: X-Forwarded-For
  - http-log:
      enabled: no
      filename: http.log
      append: yes
  - tls-log:
      enabled: no
      filename: tls.log
      append: yes
  - tls-store:
      enabled: no
  - dns-log:
      enabled: no
      filename: dns.log
      append: yes
  - pcap-log:
      enabled: no
      filename: log.pcap
      limit: 1000mb
      max-files: 2000
      mode: normal
      use-stream-depth: no
      honor-pass-rules: no
  - alert-debug:
      enabled: no
      filename: alert-debug.log
      append: yes
  - alert-prelude:
      enabled: no
      profile: suricata
      log-packet-content: no
      log-packet-header: yes
  - stats:
      enabled: yes
      filename: stats.log
      totals: yes
      threads: no
  - syslog:
      enabled: no
      facility: local5
  - drop:
      enabled: yes
      filename: drop.log
      append: yes
  - file-store:
      enabled: yes
      log-dir: files
      force-magic: no
      force-filestore: no
  - file-log:
      enabled: yes
      filename: files-json.log
      append: yes
      force-magic: no
  - tcp-data:
      enabled: no
      type: file
      filename: tcp-data.log
  - http-body-data:
      enabled: no
      type: file
      filename: http-data.log
  - lua:
      enabled: no
      scripts:
logging:
  default-log-level: notice

  default-output-filter:

  outputs:
  - console:
      enabled: yes
  - file:
      enabled: yes
      level: info
      filename: /var/log/suricata/suricata.log
  - syslog:
      enabled: no
      facility: local5
      format: "[%i] <%d> -- "
af-packet:
  - interface: eth0
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: yes
  - interface: default

pcap:
  - interface: eth0
  - interface: default

pcap-file:
  checksum-checks: auto

app-layer:
  protocols:
    tls:
      enabled: yes
      detection-ports:
        dp: 443
    dcerpc:
      enabled: yes
    ftp:
      enabled: yes
    ssh:
      enabled: yes
    smtp:
      enabled: yes
      mime:
        decode-mime: yes
        decode-base64: yes
        decode-quoted-printable: yes
        header-value-depth: 2000
        extract-urls: yes
        body-md5: no
      inspected-tracker:
        content-limit: 100000
        content-inspect-min-size: 32768
        content-inspect-window: 4096
    imap:
      enabled: detection-only
    msn:
      enabled: detection-only
    smb:
      enabled: yes
      detection-ports:
        dp: 139, 445
    nfs:
      enabled: no
    dns:
      tcp:
        enabled: yes
        detection-ports:
          dp: 53
      udp:
        enabled: yes
        detection-ports:
          dp: 53
    http:
      enabled: yes
      request-body-limit: 0
      response-body-limit: 0
      libhtp:
         default-config:
           personality: IDS
           request-body-limit: 100kb
           response-body-limit: 100kb
           request-body-minimal-inspect-size: 32kb
           request-body-inspect-window: 4kb
           response-body-minimal-inspect-size: 40kb
           response-body-inspect-window: 16kb
           response-body-decompress-layer-limit: 2
           http-body-inline: auto
           double-decode-path: no
           double-decode-query: no
         server-config:
    modbus:
      enabled: no
      detection-ports:
        dp: 502
      stream-depth: 0
    dnp3:
      enabled: no
      detection-ports:
        dp: 20000
    enip:
      enabled: no
      detection-ports:
        dp: 44818
        sp: 44818
    ntp:
      enabled: no
asn1-max-frames: 256
coredump:
  max-dump: unlimited
host-mode: auto
unix-command:
  enabled: yes
  filename: custom.socket
legacy:
  uricontent: enabled
engine-analysis:
  rules-fast-pattern: yes
  rules: yes
pcre:
  match-limit: 3500
  match-limit-recursion: 1500
host-os-policy:
  windows: [0.0.0.0/0]
  bsd: []
  bsd-right: []
  old-linux: []
  linux: []
  old-solaris: []
  solaris: []
  hpux10: []
  hpux11: []
  irix: []
  macos: []
  vista: []
  windows2k3: []
defrag:
  memcap: 32mb
  hash-size: 65536
  trackers: 65535
  max-frags: 65535
  prealloc: yes
  timeout: 60
flow:
  memcap: 128mb
  hash-size: 65536
  prealloc: 10000
  emergency-recovery: 30
vlan:
  use-for-tracking: true
flow-timeouts:
  default:
    new: 30
    established: 300
    closed: 0
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 0
    emergency-bypassed: 50
  tcp:
    new: 60
    established: 600
    closed: 60
    bypassed: 100
    emergency-new: 5
    emergency-established: 100
    emergency-closed: 10
    emergency-bypassed: 50
  udp:
    new: 30
    established: 300
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
  icmp:
    new: 30
    established: 300
    bypassed: 100
    emergency-new: 10
    emergency-established: 100
    emergency-bypassed: 50
stream:
  memcap: 64mb
  checksum-validation: yes
  inline: auto
  reassembly:
    memcap: 256mb
    depth: 0
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
 host:
  hash-size: 4096
  prealloc: 1000
  memcap: 32mb
decoder:
  teredo:
    enabled: true
detect:
  profile: medium
  custom-values:
    toclient-groups: 3
    toserver-groups: 25
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
  prefilter:
    default: mpm
  grouping:
  profiling:
    grouping:
      dump-to-disk: false
      include-rules: false
      include-mpm-stats: false
mpm-algo: auto
spm-algo: auto
threading:
  set-cpu-affinity: no
  cpu-affinity:
    - management-cpu-set:
        cpu: [ 0 ]
    - receive-cpu-set:
        cpu: [ 0 ]
    - worker-cpu-set:
        cpu: [ "all" ]
        mode: "exclusive"
        prio:
          low: [ 0 ]
          medium: [ "1-2" ]
          high: [ 3 ]
          default: "medium"
  detect-thread-ratio: 1.0
luajit:
  states: 128
profiling:
  rules:
    enabled: yes
    filename: rule_perf.log
    append: yes
    limit: 10
    json: yes
  keywords:
    enabled: yes
    filename: keyword_perf.log
    append: yes
  rulegroups:
    enabled: yes
    filename: rule_group_perf.log
    append: yes
  packets:
    enabled: yes
    filename: packet_stats.log
    append: yes
    csv:
      enabled: no
      filename: packet_stats.csv
  locks:
    enabled: no
    filename: lock_stats.log
    append: yes
  pcap-log:
    enabled: no
    filename: pcaplog_stats.log
    append: yes
nfq:
nflog:
  - group: 2
    buffer-size: 18432
  - group: default
    qthreshold: 1
    qtimeout: 100
    max-size: 20000
capture:
netmap:
 - interface: eth2
 - interface: default
pfring:
  - interface: eth0
    threads: 1
    cluster-id: 99
    cluster-type: cluster_flow
  - interface: default
ipfw:
napatech:
    hba: -1
    use-all-streams: yes
    streams: ["0-3"]
mpipe:
  load-balance: dynamic
  iqueue-packets: 2048
  inputs:
  - interface: xgbe2
  - interface: xgbe3
  - interface: xgbe4
  stack:
    size128: 0
    size256: 9
    size512: 0
    size1024: 0
    size1664: 7
    size4096: 0
    size10386: 0
    size16384: 0
cuda:
  mpm:
    data-buffer-size-min-limit: 0
    data-buffer-size-max-limit: 1500
    cudabuffer-buffer-size: 500mb
    gpu-transfer-size: 50mb
    batching-timeout: 2000
    device-id: 0
    cuda-streams: 2

And finally, let's fix the rights.


sudo mkdir /var/run/suricata
sudo chown cuckoo:cuckoo /var/run/suricata
sudo chown -R cuckoo:cuckoo /etc/suricata
sudo chown -R cuckoo:cuckoo /var/log/suricata
sudo touch /etc/suricata/threshold.config

ETupdate


Configure AutoUpdate Suricata Community Signatures.


cd /opt
sudo git clone https://github.com/seanthegeek/etupdate.git
sudo cp etupdate/etupdate /usr/sbin
sudo /usr/sbin/etupdate -V
sudo crontab -e
0 0 * * * /usr/sbin/etupdate -V

Snort


I did not add rules to it, since I trust Suricata more.


sudo apt install snort -y

When configuring, we specify the default interface and subnet 192.168.0.0/16.


sudo chown -R cuckoo:cuckoo /etc/snort/
sudo chown -R cuckoo:cuckoo /var/log/snort/

VMcloak and Windows 7 SP1 X64


The most convenient program for the automatic deployment of VM Cuckoo. It significantly reduces the time, but the author rewrote its logic of work and of course did not update the documentation. I advise you to look at either the utility code on GitHub or read the old documentation to understand all the delights of working with VMcloak and to understand what we will do next.
The most attentive harazhitel will notice that the official repository of the project https://github.com/jbremer/vmcloak differs from that used below in the script https://github.com/tweemeterjop/vmcloak . In relation to the fork, the original did not know how to include vRDE - the implementation of the RDP protocol in a custom guest VM.
Probably the correct option is to take the original code and finish vRDE into it, however, if the fork has not gone so far from the original, you can use the fork, as I do below. At the time of setup, the repository and its fork had practically no differences.
From my own experience I’ll tell you - vRDE is needed, there are use cases when the document of the target attack is protected with a Word password and the password must be entered for the virus to work in the sandbox, or the virus is stuck with the installer that needs to be installed. Without vRDE, this is problematic. And for the further configuration of the vRDE OS, we will also need it.


cd /opt
sudo apt install libyaml-dev libpython2.7-dev genisoimage -y
sudo git clone -b vrde https://github.com/tweemeterjop/vmcloak.git
cd vmcloak/
sudo cp /home/cuckoo/.cuckoo/agent/agent.py vmcloak/data/bootstrap/
sudo -H pip install -r requirements.txt
sudo python setup.py install
cd ..
sudo mkdir -p /mnt/win7
sudo mount -o loop,ro ~/en_windows_7_enterprise_with_sp1_x64_dvd_u_677651.iso /mnt/win7/
sudo vmcloak-vboxnet0
sudo vmcloak-iptables 192.168.56.0/24 ens160
cd /home/cuckoo
su cuckoo
vmcloak init --vrde --resolution 1280x1024 --ramsize 4096 --win7_x64 --product professional --cpus 2 win7x64
vmcloak install --vrde win7_x64 python27 pillow adobepdf chrome cuteftp dotnet40 flash java silverlight vcredist wic
vmcloak modify --vrde win7_x64

Configure Windows 7


After


vmcloak modify --vrde win7_x64

VM starts, you can connect to it via RDP. If you work with Win10 - do not use the tiled application of the remote desktop, it falls into a stupor and trance, when the connection goes to the VM without a username and password, use the classic application, everything is OK there.


The following items on my list are optional, however, in order to reduce the number of false positives and simplify the work of malware - it is recommended to disable (or enable) all of the following. It will not be superfluous to install software in the guest VM, which is used by all employees at work.


Example of extra data in the report:


False Positive


I will notice! All of the following should be done only for Windows 7 SP1 x64, on other OS the actions may be different.


After some time working with the new sandbox, my list began to include the following items:


  • Disable Windows Messenger broadcasts on UDP port 1900


    HKEY_LOCAL_MACHINE \ Software \ Microsoft \ DirectPlayNATHelp \ DPNHUPnP Key Name: UPnPMode Type: REG_DWORD Value: 2


  • Disable x64 driver verification so that Cuckoo works properly with x64


    cmd - bcdedit.exe / set nointegritychecks ON


  • disable NCSI


    HKEY_LOCAL_MACHINE \ SYSTEM \ CurrentControlSet \ Services \ NlaSvc \ Parameters \ Internet key EnableActiveProbing Type: REG_DWORD Value: 0


  • disable teredo.ipv6.microsoft.com


    cmd - netsh interface teredo set state disabled


  • disable IPv6


    HKEY_LOCAL_MACHINE \ SYSTEM \ CurrentControlSet \ Services \ Tcpip6 \ Parameters \ add the key DWORDDisabled DisabledComponents 8


  • Disable in Tredo network adapters
  • Disable autoload Adobe Reader, Flash player, MS Office
  • Disable Chrome Update
  • In chrome: // plugins allow launch of vulnerable flash
  • In Chrome settings, allow everything in content settings
  • In the HKEY_LOCAL_MACHINE \ SOFTWARE \ Policies \ Google branch, delete Update
  • Disable FP, UPnP, SSDP services
  • Set chrome as default browser
  • Set an empty chrome start page
  • Disable Chrome privacy settings
  • Install FileZilla
  • Turn off synchronization with NTP
  • Disable secure view office (Data execution prevention mode)
  • Allow autorun macros in all applications in the office

The sequence of compiling your own instructions for your own OS:


  1. Checking the legitimate executable in the sandbox;
  2. We look where and in what tabs the extra info appeared, for example, an appeal to time.microsoft.com;
  3. We open Google and see how we can disable it;
  4. We write, turn it off.

Adding VM to Cuckoo


We create a snapshot of the freshly configured guest OS and add to Cuckoo at the same time removing the preinstalled VM cuckoo1 from the config.


vmcloak snapshot win7_x64 win7_x64node1 192.168.56.101
. /home/cuckoo/cuckoo/bin/activate
cuckoo machine --add win7x64node1 192.168.56.101 --platform windows --snapshot vmcloak
cuckoo machine --delete cuckoo1
deactivate

Moloch


The coolest tool for working with network traffic. Unlike other network analyzers - the bet in Moloch is made on the analytics toolkit with a bunch of features, timelines, graphs and more. I am very sorry that I learned about it so recently and used the less convenient MalCom before that . One of the key benefits of Moloch is the ability to combine and analyze all the network packets of all scans from Cuckoo at once. Well, working in conjunction with Elasticsearch allows Moloch to process a huge amount of data.


This is how this beauty looks (screen from the official repository):


Moloch ГШ


Well, we install it too.


sudo apt install libjson-perl -y
cd /opt
sudo wget https://files.molo.ch/builds/ubuntu-16.04/moloch_0.20.2-2_amd64.deb
sudo dpkg -i moloch_0.20.2-2_amd64.deb
sudo /data/moloch/bin/Configure

Next, you need to specify the vboxnet0 interface, the Moloch login and password, the Elasticsearch IP address.


sudo /data/moloch/db/db.pl http://localhost:9200 init
sudo /data/moloch/bin/moloch_add_user.sh cuckoo cuckoo cuckoosandbox --admin

InetSim


We install a virtual local network so that the virus is not alone in the sandbox. The settings for emulated services were left default, 18 emulated services turned out, not including the web.


sudo su
echo "deb http://www.inetsim.org/debian/ binary/" > /etc/apt/sources.list.d/inetsim.list
wget -O - http://www.inetsim.org/inetsim-archive-signing-key.asc | apt-key add -
apt update
apt install inetsim
exit

In the /etc/inetsim/inetsim.conf config, you need to comment out the HTTP and HHTPS web services so that they do not conflict with Cuckoo:


#start_service http
#start_service https

IRMA


IRMA


IRMA is not integrated with the sandbox due to the problem of uploading reports to Elasticsearch. But this does not prevent us from deploying and testing the utility. IRMA is compiled in Vagrant and deployed automatically through Ansible strictly version 2.2.1.0.


sudo apt install vagrant -y
cd /opt
sudo wget https://releases.hashicorp.com/vagrant/2.0.2/vagrant_2.0.2_x86_64.deb
sudo dpkg -i vagrant_2.0.2_x86_64.deb
sudo -H pip install -U ansible==2.2.1.0

Download IRMA:


sudo git clone https://github.com/quarkslab/irma
cd irma/ansible

Add a redirect from VM to port 8080 by adding
line 19 to the / opt / irma / ansible / Vagrantfile file
:


config.vm.network "forwarded_port", guest: 80, host: 8080

And install IRMA:


sudo vagrant up

Honeyd


HoneyD is the oldest and most functional hanipot. There is only one drawback of it - it stopped updating several years ago. Skills for finding suitable forks were applied successfully, after which a relatively fresh version of installing HoneyD through Ansible was found , which I hastened to use. In the Hanipot, I configured only one host, you can configure at least an entire subnet using the official manual .


cd /opt/
sudo git clone https://github.com/Bifrozt/honeyd-ansible.git
cd honeyd-ansible/
sudo ansible-playbook honeyd.yml
sudo touch /usr/share/honeyd/config.conf

Save the config.


/usr/share/honeyd/config.conf
create default
set default default tcp action filtered
set default default udp action filtered
set default default icmp action filtered

create windows
set windows personality "Microsoft Windows XP Professional SP3"
set windows uptime 1728650
set windows maxfds 35
set windows default tcp action reset
add windows tcp port 135 open
add windows tcp port 139 open
add windows tcp port 445 open
set windows ethernet "08:00:27:81:1d:0c"
bind 192.168.56.103 windows

Web server


Install Nginx, generate a certificate, and even enable http2 support.


sudo add-apt-repository ppa:nginx/development
sudo apt update
sudo apt install nginx -y
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
sudo mkdir /etc/nginx/ssl
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
sudo -H pip install -U uwsgi
cd /home/cuckoo/
sudo mkdir /var/log/uwsgi/
sudo mkdir /etc/uwsgi
sudo chown cuckoo:cuckoo /var/log/uwsgi/
sudo chown cuckoo:cuckoo /etc/uwsgi/
su cuckoo

Next, add the UWSGI config.


/etc/uwsgi/cuckoo.ini
[uwsgi]
plugins = python
socket = /tmp/uwsgi.sock
chmod-socket = 664
master = true
processes = 4
virtualenv = /home/cuckoo/cuckoo
module = cuckoo.web.web.wsgi
uid = cuckoo
gid = cuckoo
static-map = /static=/home/cuckoo/cuckoo/local/lib/python2.7/site-packages/cuckoo/web/static
env = CUCKOO_APP=web
env = CUCKOO_CWD=/home/cuckoo/.cuckoo

And the nginx config, changing the IP and site name:


/ etc / nginx / sites-available / cuckoo-web
upstream _uwsgi_cuckoo_web {
    server unix:/tmp/uwsgi.sock;
}

server {
 listen 80;
 listen [::]:80;

 server_name cuckoo.test.ru;
 return 301 https://10.0.0.3$request_uri;

 server_tokens off;
}

server {
 listen 443 ssl http2;
 listen [::]:443 ssl http2;

 server_name cuckoo.test.ru;

 ssl_certificate /etc/nginx/ssl/nginx.crt;
 ssl_certificate_key /etc/nginx/ssl/nginx.key;
 ssl_dhparam /etc/ssl/certs/dhparam.pem;
 ssl_session_timeout 1d;
 ssl_session_cache shared:SSL:50m;
 ssl_session_tickets off;
 ssl_protocols TLSv1.2;
 ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
 ssl_prefer_server_ciphers on;
 add_header Strict-Transport-Security "max-age=15768000";
 ssl_stapling on;
 ssl_stapling_verify on;

 server_tokens off;

 location / {
    client_max_body_size 1G;
    proxy_redirect off;
    proxy_set_header X-Forwarded-Proto $scheme;
    uwsgi_pass  _uwsgi_cuckoo_web;
    include     uwsgi_params;
    }
}

Now we will add the www-data user to the cuckoo group and enable our config by symlink.


sudo adduser www-data cuckoo
sudo ln -s /etc/nginx/sites-available/cuckoo-web /etc/nginx/sites-enabled/
sudo systemctl reload nginx

Configs cuckoo


And, finally, the most important thing - we combine everything collected in the sandbox. To explain what’s going on for a long time, it’s pointless and 85% of the parameters are intuitive, I suggest using the official documentation starting from page 96, if there are difficulties with understanding some parameters. Or the main thing is to peep in my last article. Not the worst option would be to copy and use my files below, but at your own peril and risk.


/home/cuckoo/.cuckoo/conf/auxiliary.conf
[sniffer]
enabled = yes
tcpdump = /usr/sbin/tcpdump
bpf = 
[mitm]
enabled = yes
mitmdump = /usr/local/bin/mitmdump
port_base = 50000
script = mitm.py
certificate = bin/cert.p12
[services]
enabled = yes
services = honeyd
timeout = 0
[reboot]
enabled = yes

/home/cuckoo/.cuckoo/conf/cuckoo.conf
[cuckoo]
version_check = yes
delete_original = no
delete_bin_copy = no
machinery = virtualbox
memory_dump = yes
terminate_processes = no
reschedule = no
process_results = yes
max_analysis_count = 0
max_machines_count = 0
max_vmstartup_count = 10
freespace = 1024
tmppath = 
rooter = /tmp/cuckoo-rooter
[feedback]
enabled = no
name = 
company = 
email = 
[resultserver]
ip = 192.168.56.1
port = 2042
force_port = no
upload_max_size = 134217728
[processing]
analysis_size_limit = 134217728
resolve_dns = yes
sort_pcap = yes
[database]
connection = postgresql://cuckoo:Supersecretpassword4habr@localhost:5432/cuckoo
timeout = 60
[timeouts]
default = 120
critical = 60
vm_state = 60

/home/cuckoo/.cuckoo/conf/memory.conf
[basic]
guest_profile = Win7SP1x64
delete_memdump = yes
[malfind]
enabled = yes
filter = yes
[apihooks]
enabled = no
filter = yes
[pslist]
enabled = yes
filter = no
[psxview]
enabled = yes
filter = no
[callbacks]
enabled = yes
filter = no
[idt]
enabled = yes
filter = no
[timers]
enabled = yes
filter = no
[messagehooks]
enabled = no
filter = no
[getsids]
enabled = yes
filter = no
[privs]
enabled = yes
filter = no
[dlllist]
enabled = yes
filter = yes
[handles]
enabled = yes
filter = yes
[ldrmodules]
enabled = yes
filter = yes
[mutantscan]
enabled = yes
filter = yes
[devicetree]
enabled = yes
filter = yes
[svcscan]
enabled = yes
filter = yes
[modscan]
enabled = yes
filter = yes
[yarascan]
enabled = yes
filter = yes
[ssdt]
enabled = yes
filter = yes
[gdt]
enabled = yes
filter = yes
[sockscan]
enabled = yes
filter = no
[netscan]
enabled = no
filter = no
[mask]
enabled = no
pid_generic =

/home/cuckoo/.cuckoo/conf/processing.conf
[analysisinfo]
enabled = yes
[apkinfo]
enabled = no
decompilation_threshold = 5000000
[baseline]
enabled = no
[behavior]
enabled = yes
[buffer]
enabled = yes
[debug]
enabled = yes
[droidmon]
enabled = no
[dropped]
enabled = yes
[dumptls]
enabled = yes
[extracted]
enabled = yes
[googleplay]
enabled = no
android_id = 
google_login = 
google_password = 
[memory]
enabled = yes
[misp]
enabled = no
url = 
apikey = 
maxioc = 100
[network]
enabled = yes
whitelist_dns = no
allowed_dns = 
[procmemory]
enabled = yes
idapro = no
extract_img = no
extract_dll = no
dump_delete = no
[procmon]
enabled = yes
[screenshots]
enabled = yes
tesseract = /usr/bin/tesseract
[snort]
enabled = yes
snort = /usr/sbin/snort
conf = /etc/snort/snort.conf
[static]
enabled = yes
pdf_timeout = 60
[strings]
enabled = yes
[suricata]
enabled = yes
suricata = /usr/bin/suricata
conf = /etc/suricata/suricata-cuckoo.yaml
eve_log =  eve.json
files_log = files-json.log
files_dir = files
socket = 
[targetinfo]
enabled = yes
[virustotal]
enabled = yes
timeout = 60
scan = yes
key = тут ключ virustotal
[irma]
enabled = no
timeout = 300
scan = yes
force = yes
url = http://172.16.1.30

/home/cuckoo/.cuckoo/conf/reporting.conf
[feedback]
enabled = no
[jsondump]
enabled = yes
indent = 4
calls = yes
[singlefile]
enabled = yes
html = no
pdf = yes
[misp]
enabled = no
url = 
apikey = 
mode = maldoc ipaddr hashes url
[mongodb]
enabled = yes
host = 127.0.0.1
port = 27017
db = cuckoo
store_memdump = yes
paginate = 100
username = 
password = 
[elasticsearch]
enabled = yes
hosts = 127.0.0.1
timeout = 300
calls = no
index = cuckoo
index_time_pattern = yearly
cuckoo_node = 
[moloch]
enabled = yes
host = 10.0.0.3:8005
insecure = no
moloch_capture = /data/moloch/bin/moloch-capture
conf = /data/moloch/etc/config.ini
instance = cuckoo
[notification]
enabled = no
url = 
identifier = 
[mattermost]
enabled = no
url = 
myurl = 
username = cuckoo
show_virustotal = no
show_signatures = no
show_urls = no
hash_filename = no
hash_url = no

/home/cuckoo/.cuckoo/conf/routing.conf
[routing]
route = none
internet = ens160
rt_table = main
auto_rt = yes
drop = no
[inetsim]
enabled = yes
server = 192.168.56.1
[tor]
enabled = yes
dnsport = 5353
proxyport = 9040
[vpn]
enabled = no
vpns = vpn0
[vpn0]
name = vpn0
description = Spain, Europe
interface = tun0
rt_table = tun0

/home/cuckoo/.cuckoo/conf/virtualbox.conf
[virtualbox]
mode = headless
path = /usr/bin/VBoxManage
interface = vboxnet0
machines = win7_x64node1
[win7_x64node1]
label = win7_x64node1
platform = windows
ip = 192.168.56.101
snapshot = vmcloak
interface = 
resultserver_ip = 192.168.56.1
resultserver_port = 2042
tags = 
options = 
osprofile = Win7SP1x64
[honeyd]
label = honeyd
platform = linux
ip = 192.168.56.103
tags = service, honeyd
options = nictrace noagent

Cuckoo startup and services


Now we will make sure that our service after each power outage gets up on its own and without unnecessary help.


Install supervisor and add the Cuckoo autorun configuration to it.


sudo apt install supervisor -y
sudo systemctl stop supervisor

Create supervisor files:


/etc/supervisor/conf.d/vmcloak-internet.conf
[program:vmcloak-vboxnet0]
command=vmcloak-vboxnet0
directory=/usr/local/bin/
[program:vmcloak-ifconfig]
command=vmcloak-iptables 192.168.56.0/24 ens160
directory=/usr/local/bin/
[group:vmcloak-internet]
programs = vmcloak-vboxnet0, vmcloak-ifconfig

/etc/supervisor/conf.d/cuckoo.conf
[program:cuckoo-rooter]
command = /home/cuckoo/cuckoo/bin/cuckoo rooter --sudo
autorestart = true
[program:cuckoo-daemon]
command = /home/cuckoo/cuckoo/bin/cuckoo -d -m 10000
user = cuckoo
startsecs = 30
autorestart = true
[program:cuckoo-process]
command = /home/cuckoo/cuckoo/bin/cuckoo process p%(process_num)d
process_name = cuckoo-process_%(process_num)d
numprocs = 4
user = cuckoo
autorestart = true
[group:cuckoo]
programs = cuckoo-rooter, cuckoo-daemon, cuckoo-process
[program:distributed]
command = /home/cuckoo/cuckoo/bin/python -m cuckoo.distributed.worker
user = cuckoo
autostart = false
autorestart = true
environment = CUCKOO_APP="worker",CUCKOO_CWD="/home/cuckoo/.cuckoo"

/etc/supervisor/conf.d/uwsgi.conf
[program:uwsgi]
user = cuckoo
directory = /usr/bin
command = bash -c 'sleep 5 && uwsgi --ini /etc/uwsgi/cuckoo.ini'
autorestart = true
stderr_logfile = /var/log/uwsgi/uwsgi-err.log
stdout_logfile = /var/log/uwsgi/uwsgi-out.log
stopsignal = QUIT

And apply:


sudo systemctl restart supervisor
sudo supervisorctl -c /etc/supervisor/supervisord.conf reload

We create a script to launch the remaining services (it was possible to configure through supervisor, but there was some kind of bug that did not allow me to apply that elegant solution):


/opt/serv.sh
sleep 30
sudo systemctl start molochcapture.service
sudo systemctl start molochviewer.service
sudo inetsim
cd /opt/irma/ansible/
sudo vagrant up
sudo honeyd -f /usr/share/honeyd/config.conf -i vboxnet0

And add to the crowns:


sudo chmod +x serv.sh
sudo crontab -e 
@reboot /bin/sh /opt/serv.sh

Is there life after the cuckoo? or "Gathered a cuckoo! And then what to do?"


Gathered a new IB service from scratch at work and don’t know what to do?
The simplest answer is when the cuckoo is assembled - you need to analyze the suspicious software found with your pens and be well done.
For a small office, this is a perfectly acceptable solution, for a student of an information security university - a very acceptable solution. But for companies more, alas, no. And most likely people from larger companies are asking this question.
My advice to you is to build from scratch or modernize the business process of monitoring and responding to information security incidents, making incident processing fully automatic. This is the most direct way to improve the speed and quality of the work of the IS department or IS groups, therefore, to reduce costs and reduce risks.


Here is the bare minimum to start - there are only 3 classes of software and almost everything can be found with open source:


  1. IB system. That generates or collects data, for example, Cuckoo Sandbox, IDS, UEBA, USM, SIEM and so on;
  2. Incident Response Platform (IRP);
  3. Cyber ​​Intelligence Systems. (Threat intelligense platform, TIP).

One of the best open source information security systems that amazes me with its simplicity, usability and functionality is the TheHive Project IRP stack . The stack itself consists of several elements, we are interested in 2.


  1. TheHive is an incident response platform.
  2. Cortex is an incident data enrichment system.

TheHive Project stack.
TheHive


Thehive is an incident response platform that has an impressive architecture and amazing flexibility under the hood that allows you to adapt the incident response process to any already established business process in the company. TheHive is designed to manage incidents throughout the life cycle. Each incident (case) is divided into tasks, for example, the "DDoS site" case is divided into tasks:


  • "Determine the level, L7 or channel clog?"
  • "Change IP applications"
  • "Blackhole routing"

A report is written for each task, each task is assigned personally to the person responsible, and when the task is closed, the director of the information security division decides whether or not this information can be shared with another company office, according to TLP rules or with whom else.


TheHive UI.
TheHive UI


TheHive receives data from any system through the TheHive4Py API , for example, from SIEM, or is manually driven in by a security guard. Each incident has an important component - indicators of compromise. These are links, IPs, domains, samples of malicious files, hashes, process names, and so on. In fact, the security guard considered the threat. And here the second element of the stack appears - Cortex . He, using the API of dozens of services (commercial and not very) enriches the incident data.
Here is an example case - an antivirus on a working computer finds a suspicious file, a security guard enters the file into TheHive and logs the incident (Case). The incident is divided into tasks (Tasks), for example, "Disconnect the workstation from the network", "Check Petrov's work mail" and so on. And while the IT employee is engaged in these tasks, the information security analyst using Cortex will determine the number of antiviruses that detected the file on VirusTotal, a malicious campaign in which the file took part in AlienVault OTX , the exact date of the malicious company, file metadata, sends the file to Cuckoo Sandbox and will carry out another 8-10 checks. By pressing one button. Yes, just one button. And Cortex will do the rest and add the data to TheHive.


Misp
MISP


Data on closed confirmed incidents from TheHive and Cuckoo goes to Misp and is immediately sent to geographically remote divisions of the company, where there is a preventive blocking of compromise indicators in information protection systems: the IP address of the virus management server on the Firewall, and the antivirus hash file is blocked on the workstation.
In addition, you have a structured database of all incidents, samples of all malicious files, and many times more detailed information about the incident, collected instantly. And the built-in mechanisms of TheHive metrics will allow you to analyze the work of the monitoring and response groups to incidents, subsequently marking them a fair KPI.


MISP, among other things, has 47 predefined feeds, the inclusion of which allows you to receive indicators of compromise from other CERTs, these indicators can be checked in the logs of existing IB and IT systems. For example, a domain got into MISP, and the script made by the company’s system administrator checks all links from the corporate Proxy to MISP, and if it does, it disconnects the user from the network and the threat can be prevented even before the user is infected.
And the cherry on the cake: all of the above software is open source, which is friendly with each other, well-documented and able to work with Cuckoo Sandbox. Beauty, isn't it?


Thanks for your attention!


useful links


In this section, not all useful links were written out, I wrote down on this list what was useful in my work, I knew most of them from memory or from experience, so if there is anything to add, write to the PM.