@TWMAN‎ > ‎

Introduction (介紹)



The difference between TWMAN+ & TWMAN (TWMAN+ & TWMAN 差異)

..... Remainder ..... 
 ::::: The difference between TWMAN+ with TWMAN which are :::::
(1.) TWMAN+ has applied Interval Type-2 Fuzzy Ontology  model for malware behavioral 
Identify
.
(2.) TWMAN+ has I
ntegrated 
virtualize for 
Cluster Hybrid Architecture.
I have developed the previous version of this project (TWMAN) when I worked in NCHC. Unfortunately, I have give up this job on 2011/09/28 but why I still maintain the new version of this project (TWMAN+) ? this is because I hope I can do something for security research. Therefore, 
We focus on how to detect malware behavioral by system 
performance
 and Type-2 Fuzzy technology.
嗯 … 適逢無聊的英國聖誕夜以及年關將近 … 來個中英文摘要說明一下接著的開發方向 … 希望讓有意願參與的單位能比較清楚一點。

嗯 .. 歡迎轉載 … 但記得保留一下原文連結哦 ! XD 好歹這些想法殺了我不少腦細胞嘛 …

傳統的知識本體很明顯是不足夠處理模糊和不精確的應用在惡意程式行為分析,但惡意程式所會造成的傷害卻已是政府機關或商業公司不得忽視的重要問題也是資訊安全領域的重要研究議題。為了盡量的降低惡意程式所造成的損害,加強分析惡意程式分析技術來預測其惡意行為是很有效的方法;使用虛擬機技術正是目前最常見的方法,但惡意程式反制虛擬機的技術卻也道高一尺魔高一丈的抵禦搜集、分析和逆向工程等技術,導致透過虛擬機所得到的分析結果通常準確度不高。

基於上述原因,我們已經開發出臺灣惡意程式分析網(抬丸郎,TaiWan Malware Analysis Net,TWMAN)來解決虛擬技術所造成惡意程式行為分析技術所會造成的誤差,並且將第一型模糊邏輯以及模糊標記語言整合導入,但事實上,已經有許多研究證實了第一型模糊邏輯受限於其隸屬函數的精確度仍舊是不夠準確的;正因為如此,我們提出了整合區間第二型模糊邏輯以及自動化檔案分析及函式控制的機制 TWMAN+,並將其建置成可運行在Google App Engine 以及還可提供給Android設備的雲端服務,相信這樣的機制以及服務將可提升惡意程式行為分析結果的準確性及即時性。

It is widely pointed out that classical ontology is not sufficient to deal with imprecise and vague knowledge for some real world applications like malware (include Botnet, Virus, Backdoor and Trojan etc…) behavioral analysis. In addition, malware has become a growing important problem for governments and commercial organizations. Antimalware applications represent one of the most important research topic in the area of information security threat. As a consequence, enhanced systems for analyzing the behavior of malwares are needed in order to try to predict their malicious actions and minimize eventual computer damages. Many researchers try to use virtual machine (VM) system to monitor the malware behaviors, but there are many Anti-VM techniques which are used to ward off the collection, analysis, and reverse engineering features of the VM based malware analysis platform. Therefore, malware researcher would get inaccurate analysis results from VM based malware analysis platform. For this reason, we have developed Taiwan Malware Analysis Net (TWMAN) to improve the accuracy of malware behavioral analysis and which has intergraded Type I Fuzzy Logic (TIFL), ontology and Fuzzy Markup Language (FML). TWMAN was based on Type I Fuzzy Ontology model and which focuses on using real operation system environment to analysis malware behavioral. Indeed, there are many research has shown that there are limitations in the ability of T1FL to model and minimize the effect of uncertainties. This is because a T1FL is certain in the sense that its membership grades are crisp values. For above reason, in this project we try to bridge this gap byInterval Type II Fuzzy Logic (IT2FL) and applied to TaiWan Malware Analysis Net which has also integrate eggdrop and glftpd and make it as a cloud service (software as a service) on Google App Engine with Python and Android:TWMAN+. We believe this system would be helpful to improve the correctness of malware analysis result and reduce the loss rate of malware analysis.

Taiwan Malware Analysis Net+(TWMAN+)Updated:

1. Capture-BAT:

2. Volatility Framework:https://www.volatilesystems.com http://code.google.com/p/volatility/

The Volatility Framework is a completely open collection of tools, implemented in Python under the GNU General Public License, for the extraction of digital artifacts from volatile memory (RAM) samples. The extraction techniques are performed completely independent of the system being investigated but offer unprecedented visibilty into the runtime state of the system. The framework is intended to introduce people to the techniques and complexities associated with extracting digital artifacts from volatile memory samples and provide a platform for further work into this exciting area of research.

3. Virtual


4. Multi Clients


5. Cluster




TWMAN

Abstract :
Malware is an important topic of security threat research. In this project, a behavioral malware analysis system TWMAN was presented. This project focuses on using real operation system (OS) environment to analysis malware behavioral. Many researchers try to use virtual machine (VM) system to monitor the malware behaviors. These malware samples will only compromise the virtual operating system or virtual machine, which cannot reflect in the real operating system or real environment. Therefore, some malware researchers don’t want their sample to be analyzed in VM environment, because the analyzer cannot much useful information in VM environment.

There are many Anti-VM techniques which are used to ward off the collection, analysis, and reverse engineering features of the VM based malware analysis platform. There are differences between these two behaviors: malware behavior in real environment and in virtual environment. Therefore, malware researcher would get inaccurate analysis results from VM based malware analysis platform. In order to retrieve correct malware behavioral information, we need flexible, adaptable, and quickly analysis environment, which could discovery malware behavioral in real operation system environment, and which can quickly restore clear operation system to analysis another malware sample. For this reason, this project developed Taiwan Malware Analysis Net(TWMAN), a real operation system environment for malware behavioral analysis and analysis report. We believe this system would be helpful to improve the correctness of malware analysis result and reduce the loss rate of malware analysis.

Introduction :

In recent years, network security events were occurred frequently. They created disasters all around the world, including internet fraud activities, and data theft, etc… Malware was the key culprit. Therefore, how to detect Malware is a very important issue for network security. Malware has the potential to harm the machine, which designed to infiltrate or damage a computer system without the owner’s informed consent (e.g., viruses, backdoors, spyware, Trojans and worms)[1].

Malware Analysis :

The proliferation of malware continues to grow up at a staggering rate. It is estimated that 250 new variants of malware introduced into the world every day [4].Malwares are used to compromise and steal the users private data by the vulnerabilities of exploiting software. In the last several years, Internet malware attacks have grown up rapidly. Especially in 2008, the malware attack becomes more and more serious[5].

Up to the present, there are only two methods for malware behavioral analysis. One is the static analysis (code analysis). The other one is dynamic analysis (malware behavioral analysis), which can analyzes the network traffic of malware behavior and monitors the infected system to find out the changed files or registers.

In addition, some of the malware has been found that they exhibit the similar behavioral patterns, such as the usage of specific rules or modifications of particular system files [7, 8].Malware behavioral analysis can determine the behavior of malware. Although this technique have become more and more popular, the anti-detection technique of malware still grows up rapidly[9]. Behavioral analysis technique can be applied to monitor the behavior of the malware that infects your computer system by network traffic. [10]. Malware behavioral analysis techniques have focused on obtaining reliable and accurate information on execution of malicious programs previously [11].

Although, many malware behavioral analysis have been developed by the software companies, such as the Norman Sandbox, Virus Total and Threat Expert[12, 7, 13, 8], some malware behavior still cannot be detected for fractional exceptional malware. The reason is that those malware can distinguish that the environment they stay in is a virtual or real environment. If they find out they stay in the virtual environment, they will try to obfuscate the monitor, and this mechanism will make the analysis result to be a fault report. Making the virtual machine to crash and detecting the existence of virtual environment are two main techniques to evade the analysis of VM based analysis.

We developed a real operation system (OS) environment to analysis malware behavioral, named Taiwan Malware Analysis Net (TWMAN). In the following, we will focus on how to use this real OS environment to analysis malware behavioral and describe the system structure of TWMAN briefly. In order to verify the analysis result obtained from TWMAN is more correct, it is compared with that from sandboxs, which are VM-based and Real OS analysis technique with CWSandbox of Sunbelt Software.

Don’t worry about breaking the system when it comes time for a TWMAN update.

I probably won’t be updating it ever before Sept. 2011.

Reference :

[1] Hengli Zhao, Ming Xu, Ning Zheng, Jingjing Yao, and Q. Ho, "Malicious Executables Classification Based on Behavioral Factor Analysis," presented at the 2010 International Conference on e-Education, e-Business, e-Management and e-Learning, Sanya, China, 2010.
[2] Wikipedia. Storm botnet. Available: http://en.wikipedia.org/wiki/Storm_botnet
[3] F.-S. Corporation. F-Secure Reports Amount of Malware Grew by 100% during 2007. Available: http://www.f-secure.com/f-secure/pressroom/news/fs_news_20071204_1_eng.html
[4] J. Stewart, "Behavioural malware analysis using Sandnets," Computer Fraud & Security, vol. 2006, no. Issue, pp. 4-6, December 2006.
[5] S. Corp. Symantec Internet Security Threat Report: Trends for July-December 2007 (Executive Summary).
[6] H. D. Huang, T. Y. Chuang, Y. L. Tsai, and C. S. Lee, "Ontology-based Intelligent System for Malware Behavioral Analysis," presented at the 2010 IEEE World Congress on Computational Intelligence (WCCI2010), Barcelona, Spain, 2010.
[7] S. Software. (2007). CWSandbox user guide v 2.1.13.
[8] C. Willems, T. Holz, and F. Freiling, "Toward automated dynamic malware analysis using CWSandbox," IEEE Security & Privacy, vol. 5, no. Issue, pp. 32-39, 2007.
[9] A. Vasudevan, "MalTRAK: Tracking and Eliminating Unknown Malware," presented at the Computer Security Applications Conference, 2008. ACSAC 2008. Annual, Anaheim, CA,, 2008.
[10] G. Jacob., H. Debar., and E. Filiol., "Behavioral detection of malware: from a survey towards an established taxonomy," Journal in Computer Virology, vol. 4, no. Issue, pp. 251-266, 2008.
[11] K. Rieck, T. Holz, C. Willems, P. Düssel, and P. Laskov, "Learning and classification of malware behavior," presented at the Fifth Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA 08), 2008.
[12] J. Clausing, "Building an automated behavioral malware analysis environment using open source software," SANS Institute Reading Room2009.
[13] J. Van Randwyk, L. Ken Chiang Lloyd, and K. Vanderveen, "Farm: An automated malware analysis environment," presented at the Security Technology, 2008. ICCST 2008. 42nd Annual IEEE International Carnahan Conference on, Prague, 2008.
[14] J. Stewart. Truman – the reusable unknown malware analysis net. Available: http://www.secureworks.com/research/tools/truman.html


近年來,網路安全事件頻繁發生,造成了世界各地的災害,包括網路詐騙和資料竊取等,其中的罪魁禍首正是持續以驚人的成長速度在擴散的惡意程式,因此,藉由監測惡意程式的行為,在惡意程式執行之前即可獲取可靠和重要的訊息,以便提供進一步的防護的惡意程式行為分析是資訊安全領域非常重要的研究議題及網路安全問題;惡意程式主要是未經所有者允許即滲入或損害電腦系統(例如:病毒、後門、間諜軟體、木馬和蠕蟲等),透過軟體的漏洞來竊取用戶的私人數據,特別是在2008年,據估計,每天有250個新變種的惡意程式產生,整個網際網路惡意程式攻擊迅速的成長且變得越來越嚴重;截至目前為止,有兩種惡意程式的分析方法,一個是程式碼分析(靜態分析),另一種是應用於監測惡意程式的網路連線通訊行為,監測遭惡意程式感染的系統以及分析網路流量,並找出遭惡意改變的檔案和登錄檔,其目標是針對在惡意程式執行之前即可獲取可靠和重要的訊息的惡意程式行為分析(動態分析)。

雖然惡意程式行為分析技術已經越來越流行,一些惡意程式已被發現使用特定的規則或修改特定的系統文件等類似的行為模式,目前資訊安全領域的研究人員多透過虛擬機器來監控惡意程式的行為,但在虛擬環境中監控分析,惡意程式將只會損害到虛擬作業系統或虛擬機器,不會危害到實際的系統,雖然提升了分析判斷的速度,但卻造成無法反映出在惡意程式在真實的作業系統或真實環境中的影響,且很多惡意程式作者更不希望自己開發的惡意程式在虛擬環境中被分析,所以反惡意程式分析的技術亦迅速的成長,導致在虛擬機器內執行分析無法獲得太多有用且可靠的資訊,更發展出很多用來抵禦搜集,分析以及逆向工程的反虛擬機器偵測技術,使得惡意程式樣本在虛擬以及真實的環境中的行為將不相同,而導致惡意程式研究學者將得到不正確的結果。因此,目前雖然已有許多惡意程式行為分析的軟體開發公司,例如:Norman SandboxVirus Total以及Threat Expert等,但仍有一些惡意程式的行為無法被檢測出來,這是因為惡意程式已具備區分出所處環境的能力,如果發現是在虛擬機器環境中,便會積極的使其崩潰或混淆其分析結果,這是逃避遭虛擬機器分析的方法;基於前面所述,為了獲取正確的惡意程式行為,需要一個靈活,適應性強並能迅速分析的環境,且可以探索出惡意程式在真實作業系統中的行為,並且能快速的還原至乾淨的環境進行下一次分析。

因此我開發了臺灣惡意程式分析網(抬丸郎,TaiWan Malware Analysis NetTWMAN),一個惡意程式行為分析的真實作業系統環境,主要是使用真實的作業系統環境去分析惡意程式,並且提升惡意程式行為分析的正確性來解決虛擬技術所造成惡意程式行為分析技術所會造成的誤差,並且將第一型模糊邏輯、知識本體以及模糊標記語言整合導入;但事實上,已經有許多研究證實了第一型模糊邏輯受限於其隸屬函數的精確度仍舊是不夠準確的,加上傳統的知識本體也明顯不足處理充滿模糊和不精確語義的惡意程式行為分析應用;因此,我們提出了整合區間第二型模糊邏輯以及自動化檔案分析控制的機制TWMAN+,並將其建置成可運行在Google App Engine 以及還可提供給Android設備的雲端服務,相信這樣的機制以及服務將可提升惡意程式行為分析結果的準確性及即時性;因為惡意程式所造成的傷害卻已是政府機關或商業公司不得忽視的重要問題。為了盡量的降低惡意程式所造成的損害,加強分析惡意程式分析技術來預測其惡意行為將是很有效的方法之一。

TWMAN+整合區間第二型模糊集合及建置實體環境的惡意程式行為分析平台,主要是使用實體的作業系統環境去分析惡意程式,以便獲取更精確的惡意程式行為,提供給使用者評估,並透過其回饋機制加以提升其精確度。

TWMAN+是用來分析未知惡意程式行為的自動化惡意程式行為分析平台,其架構為Client-ServerClient端是採用實體MicrosoftWindows作業系統來進行分析,而Linux則為Server端並透過其PXE-BOOT於實體系統環境中來進行分析控制;其中,使用Linux來進行惡意行為分析控制的好處是不會受到大部份針對Windows作業系統的惡意行為感染之影響。為了進一步瞭解其惡意程式所造成的惡意行為會有什麼影響,TWMAN+在實體Windows作業系統分析環境中,透過檢測HTTPFTPSMTPIRC等網路連接之重要登錄信息及日誌,可瞭解惡意程式通常會試圖執行或安裝未知的Windows應用程式以及能對外進行網路溝通的WinSock函式、TCP/IP等網路通訊及所調用之函式;另外,也需注意其Windows作業系統中的登錄檔存儲設置,特別是一些能被惡意行為修改且能自動開機執行未知行為的登錄檔。除此之外,本計畫亦透過PythonGoogle Apps Google App Engine 建置網路應用程式;預期是希望透過多數單位的系建置(相異Client OS),輔以glftpdeggdrop達成雲端共享樣本分析技術,再經由第二型模糊推論並將資料共享;另一方面,在 Android Device也正在測試,有鑑於該平臺的惡意程式也日漸增加,預期透過TWMAN+線上送出未知應用程式的分析以及即時通知使用者樣本資訊和報告



The difference between TWMAN & Truman (TWMAN & Truman 差異)


The Reusable Unknown Malware Analysis Net(Truman)

Truman can be used to build a "sandnet", a tool for analyzing malware in an environment that is isolated, yet provides a virtual Internet for the malware to interact with. It runs on native hardware, therefore it is not stymied by malware which can detect VMWare and other VMs. The major stumbling block to not using VMs is the difficulty involved with repeatedly imaging machines for re-use. Truman automates this process, leaving the researcher with only minimal work to do in order to get an initial analysis of a piece of malware.Truman consists of a Linux boot image (originally based on Chas Tomlin’s Windows Image Using Linux) and a collection of scripts. Also provided is pmodump, a Perl-based tool to reconstruct the virtual memory space of a process from a Physical Memory dump. With this tool it is possible to circumvent most packers to perform strings analysis on the dumped malware.

The Reusable Unknown Malware Analysis Net(Truman)是由Secure Work研究員Joe Stewart為了因應越來越多未知的惡意程式日漸強大之可事先偵測環境功能,只有在實體系統環境下才會進行惡意破壞動作,使用Sandbox或虛擬機器進行分析已不敷使用,所開發之具Sandnet概念的惡意程式分析工具;Truman使用了真實的作業系統環境供惡意程式執行,並模擬常用的網路通訊協定,供惡意程式進行網路連線,最後將其感染前後的映像檔以dd儲存比對並還原,以取得惡意程式真對作業系統做了什麼惡意破壞行為,但是,Truman是2006年所開發,在現今軟硬體進步更新快速的狀況下不敷使用,加上採用dd來進行系統儲存及還原耗時不短,唯有進行軟體更新才適用。

Taiwan Malware Analysis Net(TWMAN)Updated:

1. Linux Kernel update:2.4.22 –> 2.6.31

2.6.31版的硬體支援度較2.4.22版高,包括GiGa網卡、SATA硬碟等。

2. dd –> Clonezilla Live:http://clonezilla.nchc.org.tw/clonezilla-live/

dd:a common Unix program whose primary purpose is the low-level copying and conversion of raw data.

Clonezilla Live:a small bootable GNU/Linux distribution for x86/amd64 (x86-64) based computers. Clonezilla SE (Server Edition) has been developed from 2004, and it is used to clone many computers simultaneously. It is an extremely useful tool, however, it does have several limitations. In order to use it, you must first prepare a DRBL server AND the machine to be cloned must boot from a network (e.g. PXE/Etherboot/gPXE). To address these limitations, in 2007, the Free Software Lab at the NCHC has combined Debian Live with Clonezilla to produce "Clonezilla Live," a software that can be used to easily clone individual machines. The primary benefit of Clonezilla Live is that it eliminates the need to set up a DRBL server ahead of time and the need for the computer being cloned to boot from a network. Clonezilla Live can be used to clone individual computers using a CD/DVD or USB flash drive. Though the image size is limited by the boot media’s storage capacity, this problem can be eliminated by using a network filesystem such as sshfs or samba.

最大的差異在於 dd 的速度非常慢且是完整dump,不論你硬碟使用咯多少空間,而Clonezilla Live則可以依所使用的硬碟空間進行壓縮,且速度較 dd 快很多。

3. sandnet –> InetSim:http://www.inetsim.org/

sandnet:truman作者透過perl所撰寫的虛擬網路服務,只有http、ftp、irc等常見服務且效果不好。

InetSim:a software suite for simulating common internet services in a lab environment, e.g. for analyzing the network behaviour of unknown malware samples.

inetsim比truman提供了更多常見的tcp和udp的網路服務。

Taiwan Malware Analysis Net(TWMAN)Added:

1. ssdeep:http://ssdeep.sourceforge.net/ (added)

一般在辨識惡意程式樣本時,大多使用md5編碼,但只要打開惡意程式樣本,隨便加入一個數字或字元,即會造成md5完全不一樣;ssdeep不會有這樣的情形,ssdeep 是一款計算文件片段內容以取出特徵值(CTPH)的程式。這種方式也稱為模擬特徵值。CTPH 可以比對文件內容是否相同,這型的比對有著相同順序性,以及可辨識性。如果可提供使用者比對與鑑識電腦中的檔案、文件是否有遭受惡意竄改,這類型的竄改有可能是人為的惡意修改檔案,或是病毒、木馬擅自修改程式內容。

$ cp file1.exe file2.exe

$ echo 1 >> file2.exe

$ md5sum file1.exe file2.exe

72bdd3bd37a0b5d1dd5f1be80cb29639  file1.exe

a626b78fa6ba13fdd9cfddb9f55ee7c6  file2.exe

$ ssdeep -b file1.exe file2.exe

ssdeep,1.0–blocksize:hash:hash,filename

768:my+qxlsz7yiV0+7YUaFhLFAtVI0xbM

LvzEg1B1Ki8nJ78:R+qxlsHvGhLFyI0l8tC5J78,”file1.exe”

768:my+qxlsz7yiV0+7YUaFhLFAtVI0xbM

LvzEg1B1Ki8nJ7V:R+qxlsHvGhLFyI0l8tC5J7V,”file2.exe”

以冒號分隔,第一個 (768) 是每塊單元的大小,其後兩個是檔案的 ssdeep hash 簽名 my+qxlsz7yiV0+7YUaFhLFAtVI0xbMLvzEg1B1Ki8nJ7V 和 R+qxlsHvGhLFyI0l8tC5J7V,最後便是 檔案的位置 (“file2.exe”)。這裡要看的是那兩個 ssdeep hash 簽名 - 由兩個很相似的檔案計算出來 ssdeep 簽名是非常像,除了最後一個字元 ( “8″ 對 “V” )。

2. RegRipper:http://regripper.net/

RegRipper was created and maintained by Harlan Carvey.  RegRipper, written in Perl, is the fastest, easiest, and best tool for registry analysis in forensics examinations.  RegRipper has been downloaded over 5000 times and used by examiners everywhere. Further, RegRipper is NOT intended for use with live hive files.  Hive files need to be extracted from a case (or from a live system usingFTK Imager…), or accessible via a tool such as Mount Image Pro or F-Response. RegRipper uses plugins (similar to Nessus) to access specific Registry hive files in order to access and extract specific keys, values, and data, and does so by bypassing the Win32API.

3. AIDE:http://aide.sourceforge.net/stable/manual.html

It creates a database from the regular expression rules that it finds from the config file(s). Once this database is initialized it can be used to verify the integrity of the files. It has several message digest algorithms (see below) that are used to check the integrity of the file. All of the usual file attributes can also be checked for inconsistencies. It can read databases from older or newer versions. See the manual pages within the distribution for further info.


What:實體感染、快速還原、前後比對、自由軟體。

Why:應付Timing attack、Special instructions、Fingerprint of virtual environment、Emulator bug等Anti-VM malware手法。

When:與VM分析環境同步使用,避開Anti-VM malware。

Where:實體的Client-Server環境。

Who:惡意程式行為分析。

How:自由軟體,可自己增加其它各種Forensics工具。

子網頁 (1): History (血淚史)