Module 05 Enhanced Incident Detection with Threat Intelligence
ELK is a combination of Elasticsearch, Logstash, and Kibana projects.
Lab Scenario
Threat intelligence, presented in the form of Indicators of Compromise (IoCs) or threat feeds, provides evidence-based knowledge regarding an organization’s unique threat landscape. As a SOC analyst, you should be able to identify the IoCs of malicious behavior and how to integrate them into ELK Stack.
Lab Objectives
The objective of this lab is to help students to learn how to integrate malware IoCs in ELK stack.
Lab Tasks
In this exercise you have learnt how to integrate malware IoCs in ELK stack.Exercise 2: Integrating OTX Threat Data in OSSIMThe AlienVault Open Threat Exchange (OTX) is the open threat information sharing and analysis network.
Lab Scenario
The OTX provides open access to a global community of threat researchers and security professionals. The OTX community reports on and receives threat data in the form of pulses. As a SOC analyst, you should be able to integrate OTX pulses into OSSIM to consume threat intelligence feed.
Lab Objectives
The objective of this lab is to help student to learn how to integrate OTX pulses into OSSIM.
Lab Tasks
In this exercise you have learnt how to integrate how to integrate OTX pulses into OSSIM.
ELK is a combination of Elasticsearch, Logstash, and Kibana projects.
Lab Scenario
Threat intelligence, presented in the form of Indicators of Compromise (IoCs) or threat feeds, provides evidence-based knowledge regarding an organization’s unique threat landscape. As a SOC analyst, you should be able to identify the IoCs of malicious behavior and how to integrate them into ELK Stack.
Lab Objectives
The objective of this lab is to help students to learn how to integrate malware IoCs in ELK stack.
Lab Tasks
- Click SIEM1 then, click Ctrl+Alt+Del.
- By default Administrator account is selected, type Pa$$w0rd and press Enter to login.
- Navigate to E:\SOC-Tools\ Module 05 Enhanced Incident Detection with Threat Intelligence and copy ELK Folder.
- Paste the copied ELK folder under C: drive.
- Right-click and unzip all the zip files under C:\ELK. Delete the zip files after unzipping.
- Navigate to E:\SOC-Tools\Module 05 Enhanced Incident Detection with Threat Intelligence\NSSM and copy nssm-2.24 folder.
- Paste the copied nssm-2.24 folder under C:\ELK. Close the Windows Explorer.
- To install elasticsearch service, click Windows Start button. Right-click on Windows PowerShell Command Prompt and click Run as Administrator from the context menu.
- Type Invoke-Expression -command "C:\ELK\elasticsearch-6.5.4\bin\elasticsearch-service install" and press Enter.
- You will see The service 'elasticsearch-service-x64' has been installed. in the command window.
- Type Invoke-Expression -command "C:\ELK\elasticsearch-6.5.4\bin\elasticsearch-service manager" and press enter.
- The Elasticsearch service Properties windows will open; change the startup type as Automatic and click Start to start the service. Click OK.
- You will see Successfully started service manager for 'elasticsearch-service-x64'. in the command window.
- Navigate to C:\ELK\elasticsearch-6.5.4\config folder. Right-click elasticsearch.yml file and click Edit with Notepad++ to open the file with notepad++.
- Go to Line no. 55 under Network section and remove **# ** to uncomment line. Change line network.host: 192.168.0.1 to network.host: "0.0.0.0". Click Save to save the file and close it.
- To test the elasticsearch configurations Launch Google Chrome browser, type http://localhost:9200/ and press Enter. You will see the web page as shown in following screenshot. This indicates that the elasticsearch is installed successfully.Close the Google Chrome browser.
Note: If you do not see the output check if elasticsearch service is running or not. If elasticsearch service is not running start the service and refresh the browser this may take a while. - To create logstashpipeline.conf file, go to Windows PowerShell, type notepad C:\ELK\logstash-6.5.4\bin\logstashpipeline.conf and press Enter.
- Notepad will open with alert popup message as shown in following screenshot. click Yes button.
- To add the below configuration in logstashpipeline.conf click
input {beats { port => 5044 }} output { elasticsearch { hosts => "localhost:9200" manage_template => false index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" }} - Save and close the logstashpipeline.conf file.
- To install logstash, switch to Windows PowerShell. Type Invoke-Expression -command "C:\ELK\nssm-2.24\win64\nssm install Logstash" and press Enter. The NSSM service installer will open.
- In the Application tab, click Path Textbox and type C:\ELK\logstash-6.5.4\bin\logstash.bat
- Click Startup directory and type C:\ELK\logstash-6.5.4\bin
- Click Arguments Textbox adn type -f logstashpipeline.conf
- To install Logstash service click Install service button.
- "Service “Logstash” installed Successfully!" message will appear. Click OK.
- Close all open applications and folders.
- Right-click windows start button, Click Search and type service into search field, click on Services to open Services window.
- To start Logstash service, Search and select LogStash service, click on Start in the left pane.
- You will see Logstash service will be started.
- Open Windows PowerShell, type Invoke-Expression -command "C:\ELK\nssm-2.24\win64\nssm install Kibana" command, and press Enter to install Kibana.
- NSSM service installer will open.
- In the Application tab, click Path Textbox and type C:\ELK\kibana-6.5.4-windows-x86_64\bin\kibana.bat
- Click Startup directory and type C:\ELK\kibana-6.5.4-windows-x86_64\bin
- To install Kibana service click Install service button.
- "Service “Kibana” installed Successfully!" message will appear. Click OK.
- Navigate to C:\ELK\kibana-6.5.4-windows-x86_64\config folder and right-click on kibana.yml file and click Edit with Notepad++.
- Go to Line no 7, remove **# ** to uncomment line, change the line to **server.host: “10.10.1.16” **.Click Save to save the file and close it.
- To start Kibana service, Launch Services window and select Kibana service. Click Start in left pane.
- You will see Kibana service will be started.
- Close all open applications and folders.
- Launch Google chrome browser, type http://10.10.1.16:5601 and press Enter. The Kibana home page appears as shown in following screenshot.Close the browser.
Note: If you see Kibana server is not ready yet, refresh the browser few times till you see Kibana home page. - Launch WinServer2012.
- click Ctrl+Alt+Del.
- By default Administrator account is selected, type Pa$$w0rd and press Enter to login.
- Navigate to E:\SOC-Tools\ Module 05 Enhanced Incident Detection with Threat Intelligence and copy Beats folder.
- Paste the copied Beats folder under C: Drive.
- Open Beats folder. Right-click and unzip the winlogbeat-6.5.4-windows-x86_64.zip file under C:\Beats. Delete the zip file after unzipping.
- click Windows Start button. Right-click on Windows PowerShell icon and click Run as Administrator from the context menu..
- To check connection between ELK and Endpoint type Test-NetConnection 10.10.1.16 -port 5044 and press Enter.
- You will see TcpTestSuccesseded: true in the command window.
Note: If TcpTestSuccesseded: false, then check connection between ELK Server (SIEM1) and WinServer2012. - To navigate to winlogbeat directory, type Cd C:\Beats\winlogbeat-6.5.4-windows-x86_64 ,press Enter.
- Type PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-winlogbeat.ps1 and press Enter.
Note: If Security warning message appears, type R and press Enter to continue. - You will see winlogbeat will be sucessfully installed and the status of winlogbeat service will be Stopped.
- Navigate the location C:\Beats\winlogbeat-6.5.4-windows-x86_64 and open the file winlogbeat.yml with Notepad++.
- In Kibana section under setup.kibana: Line no 71, delete **# ** to uncomment line and change host: "localhost:5601" to host: "10.10.1.16:5601".
- In Elasticsearch output section **add # ** to comment Line no 96 and 98.
- In Logstash output section under output.logstash: **delete # ** from beginning of Line no 106 and 108 to uncomment and change hosts: ["localhost:5044"] to hosts: ["10.10.1.16:5044"] .
- Save the file winlogbeat.yml and close.
- To test winlogbeat configuration, Switch to the Windows PowerShell and type .\winlogbeat.exe test config -c .\winlogbeat.yml –e and press Enter
Note: In powershell window you should be in winlogbeat-6.5.4-windows-x8664 directory before executing the above command. If not refer step no. 47 to navigate to winlogbeat-6.5.4-windows-x8664 directory. - Config Ok message will display as shown in following screenshot.
- Launch SIEM1 virtual machine.
- Right-click windows start button, Click Search and type service into search field, click on Services to open Services window.
- To Restart Logstash service, select Logstash service. Click Restart in left pane.
- To Restart Elasticsearch service, select Elasticsearch service. Click Restart in left pane.
- To Restart Kibana service, select Kibana service. Click Restart in left pane.
- To ensure Kibna service is restarted sucessfully, launch Google chrome browser, type http://10.10.1.16:5601 and press Enter. The Kibana home page appears as shown in following screenshot.Close the browser.
Note: If you see Kibana server is not ready yet, refresh the browser few times till you see Kibana home page. - Launch WinServer2012
- To load the index templet in Elasticsearch manually, open Windows PowerShell .\winlogbeat.exe setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["10.10.1.16"]' command and press Enter. You will see Loaded index templet message in the Poweshell window.
- To setup the Kibana dashboard, type .\winlogbeat.exe setup -dashboards command and press Enter. You will see Loaded dashboards message in the Poweshell window.
- Finally, to start the winlogbeat service, type Start-Service winlogbeat and press Enter.
- Close all application.
- Switch to SIEM1 virtual machine.
- Navigate to E:\SOC-Tools\Module 05 Enhance Incident Detection with Threat Intelligence and copy the malware.yml file.
- Paste the copied malware.yml file in C:\ELK\logstash-6.5.4.
- Navigate to C:\ELK\logstash-6.5.4\bin. Open the logstashpipeline.conf file with Notepad++.
- To include the filter for logstash. Press Enter after the input block [Line no. 4] and click
filter { translate { field => "message" destination => "Malware" dictionary_path => "C:\ELK\logstash-6.5.4\malware.yml" fallback => "NO" "exact" => true "regex" => true } } - Save and close the logstashpipeline.conf file.
- To Restart Logstash service, Launch Services window and select Logstash service. Click Restart in left pane.
- Launch Google Chrome browser and type http://10.10.1.16:5601 and press Enter, to open Kibana home page.
Note: Make sure Elasticsearch, Logstash, kibana services are running. - Click on Management menu in Kibana home page.To create a Index pattern, Click Index Patterns
- Type winlogbeat-* in the Index pattern field and click Next step.
Note: If you see ** There is already a index pattern with name winlogbeat-* ** message then skip to step 75. - Select I don’t want to use the Time Filter in Time Filter field name and click Create index pattern.
- You will see the created winlogbeat-* index pattern along with list of fields.
- Click Discover menu, you will see winlogbeat-* index pattern and logs associated with winlogbeat.
- Click Auto-refresh and select Refresh Interval as 1 minute.
- Type wikiworm.exe in the >_ Search..(e.g. Status:200 AND extension:PHP) textbox and click Update. You will see No result match your search criteria; leave browser open.
- Switch to WinServer2012.
- Navigate to E:\SOC-Tools\Module 05 Enhance Incident Detection with Threat Intelligence and copy the wikiworm folder.
- Paste the copied wikiworm folder in C: drive.
- Navigate to C:\wikiworm folder and double-click wikiworm.exe file. A pop-up box box appear, click OK.
- Switch to SIEM SIEM1 virtual machine, click on Refresh button in Kibana home page.You will find Event logindicating execution of wikiworm, click the arrow before the event to expand event log.You will see, the wikiworm.exe process is marked "YES" in the malware column.
- In this way, you can integrate IoCs of certain malicious behavior in ELK stack to identify it as a malicious activity.
In this exercise you have learnt how to integrate malware IoCs in ELK stack.Exercise 2: Integrating OTX Threat Data in OSSIMThe AlienVault Open Threat Exchange (OTX) is the open threat information sharing and analysis network.
Lab Scenario
The OTX provides open access to a global community of threat researchers and security professionals. The OTX community reports on and receives threat data in the form of pulses. As a SOC analyst, you should be able to integrate OTX pulses into OSSIM to consume threat intelligence feed.
Lab Objectives
The objective of this lab is to help student to learn how to integrate OTX pulses into OSSIM.
Lab Tasks
- Launch SIEM2, click Ctrl+Alt+Del.
- By default Admin account is selected, type Pa$$w0rd and press Enter to login.
- Launch Chrome browser and type https://otx.alienvault.com and press Enter to access AlienVault OTX home page.
- In the SIGN UP form, fill the required details and click on SIGN UP button to sign up for AlienVault OTX website.
Note: Please provide a working e-mail ID at the time of registration. Once the registration is done, you will receive an activation e-mail. Activate your account as instructed in the e-mail to use the tool. - After successful signing up, login to AlienVault OTX account with user name and password details.By default, the AlienVault Settings page appears. The Settings page contains the settings of your OTX account as well as OTX key and Email Notifications. Copy the OTX Key.and close the web browser.
- Launch OSSIMServer, Click Ctrl+Alt+Del. Wait for the login screen to appear. Type root as the username and press Enter key. Type toor as password and press Enter.
- AlienVault Setup screen appears as shown in the screenshot.
- Switch to SIEM2.
- Launch browser, type https://10.10.1.14, in the address bar and press Enter.Your connection is not private page appears, click Advanced.
- Click Proceed to 10.10.1.14 (unsafe) link. Here, the IP address of the OSSIM machine is 192.168.0.55.
- Administartor Account Creation page appears. Enter admin in FULL NAME field, admin@123 in PASSWORD and CONFIRM PASSWORD fields give the email address used for creating alienvault otx account and click START USING ALIENVAULT.
- Login screen appears. Enter admin in USERNAME field, admin@123 in PASSWORD field and click LOGIN.
Note: If HELP US IMPROVE ALIENVAULT OSSIM popup appears click CANCEL. - **Welcome to the AlienVault OSSIM Getting Started Wizard ** page appears. Click Skip AlienVault Wizard
- The DASHBOARD window appears.
- Navigate to **CONFIGURATION -->OPEN THREAT EXCHANGE ** menu. OPEN THREAT EXCHANGE page appear, paste the copied OTX Key into the OTX Key text box, click on CONNECT OTX ACCOUNT.
- The OTX account will be linked and user name will be seen as shown following screenshot.
- OSSIM will connect to the OTX account and start downloading the OTX pulses, this will take some time.Refresh the page and you will see that the subscribed pulses start downloading.
- Now, OSSIM will automatically detect and notify the activities on the network against the subscribed OTX pulses.
- Close all the open windows.
In this exercise you have learnt how to integrate how to integrate OTX pulses into OSSIM.