ABSTRACT
This chapter discusses Hadoop/MapReduce and describes scalable Lempel–Ziv–Welch (LZW) and quantized dictionary (QD) construction algorithm using MapReduce (MR). It presents details of the results of various MR algorithms for the construction of QD. The first MR job is dedicated for LZW dictionary construction in and the second MR job is dedicated for QD construction in. In the first MR job, The mapper takes user ID along with command sequence as an input to generate intermediate pair having the form. Recall that in single map reduce job (1MRJ) reducer will get each distinct user as key and values will be LZW dictionary pattern. The chapter also discusses the scalability of authors’ techniques and the issues in designing big data analytic techniques for insider threat detection. It examines the various techniques for stream mining for insider threat detection.
