Awesome

Welcome to spartan2

Introduction

spartan2 is a collection of data mining algorithms on big graphs and time series, providing three basic tasks: anomaly detection, forecast, and summarization. (see readthedocs, and tutorials )

Graphs and time series are fundamental representations of many key applications in a wide range of

• online user behaviors, e.g. following in social media, shopping, and downloading Apps,
• finance, e.g. stock tradings, and bank transfers,
• sensor networks, e.g. sensor readings, and smart power grid, and
• health, e.g. electrocardiogram, photoplethysmogram, and respiratory inductance plethysmography.

In practice, we find that thinking graphs and time series as matrices or tensors can enable us to find efficient (near linear), interpretable, yet accurate solutions in many applications. Therefore, our goal is developping a collectioin of algorithms on graphs and time series based on tensors (matrix is a 2-mode tensor).

In real world, those tensors are sparse, and we are required to make use of the sparsity to develop efficient algorithms. That is why we name the package as spartan: sparse tensor analytics.

The package named spartan can be imported and run independently as a usual python package. Everything in package spartan is viewed as a tensor (sparse).

Install requirements

This project requires Python 3.7 and upper. We suggest recreating the experimental environment using Anaconda through the following steps.

1. Install the appropriate version for Anaconda from here - https://www.anaconda.com/distribution/

2. Create a new conda environment named "spartan"

       conda create -n spartan python=3.7
       conda activate spartan
   

3. If you are a normal USER,

   # install spartan using pip
   pip install spartan2
   

4. If you want to contribute, or prefer to run directly on the code,

   <details> <summary>Please do the following setup</summary>

   • 4.1 Clone the project from github

     git clone git@github.com:BGT-M/spartan2.git
     

   • 4.2 Install requirements.

     # [not recommended]# pip install --user --requirement requirements
     # using conda tool
     conda install --force-reinstall -y --name spartan -c conda-forge --file requirements
     

     or use the following way

     # this may not work in ubuntu 18.04
     python setup.py install
     

   • 4.3 Install code in development mode

     # in parent directory of spartan2
     pip install -e spartan2
     

   • 4.4 Since you install your package to a location other than the user site-packages directory, you will need to add environment variable PYTHONPATH in ~/.bashrc

     export PYTHONPATH=/<dir to spartan2>/spartan2:$PYTHONPATH
     

     or prepend the path to that directory to your PYTHONPATH environment variable.

     import sys
     sys.path.append("/<dir to spartan2>/spartan2")
     

     or do as follows

     #find directory of site-packages
     python -c 'import site; print(site.getsitepackages())'
     
     #add \<name\>.pth file in your site-packages directory with string '/<dir to spartan2>/spartan2'
     
     

   </details>

Table of Modules

References

1. <span id="ref1"></span> Shenghua Liu, Bryan Hooi, and Christos Faloutsos, "HoloScope: Topology-and-Spike Aware Fraud Detection," In Proc. of ACM International Conference on Information and Knowledge Management (CIKM), Singapore, 2017, pp.1539-1548.

2. <span id="ref2"></span> Shenghua Liu, Bryan Hooi, Christos Faloutsos, A Contrast Metric for Fraud Detection in Rich Graphs, IEEE Transactions on Knowledge and Data Engineering (TKDE), Vol 31, Issue 12, Dec. 1 2019, pp. 2235-2248.

3. <span id="ref3"></span> Prakash, B. Aditya, Ashwin Sridharan, Mukund Seshadri, Sridhar Machiraju, and Christos Faloutsos. "Eigenspokes: Surprising patterns and scalable community chipping in large graphs." In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 435-448. Springer, Berlin, Heidelberg, 2010.

4. <span id="ref4"></span> Wenjie Feng, Shenghua Liu, Christos Faloutsos, Bryan Hooi, Huawei Shen, and Xueqi Cheng. EagleMine: Vision-guided Micro-clusters recognition and collective anomaly detection, Future Generation Computer Systems, Vol 115, Feb 2021, pp.236-250.

   <span id="ref4.2"></span>Wenjie Feng, Shenghua Liu, Christos Faloutsos, Bryan Hooi, Huawei Shen, Xueqi Cheng, Beyond outliers and on to micro-clusters: Vision-guided anomaly detection, In Proc. of the 23rd Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2019), 2019, Macau, China, pp541-554.

5. <span id="ref5"></span> Hooi, Bryan, Hyun Ah Song, Alex Beutel, Neil Shah, Kijung Shin, and Christos Faloutsos. "Fraudar: Bounding graph fraud in the face of camouflage." In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 895-904. 2016.

6. <span id="ref6"></span> Houquan Zhou, Shenghua Liu, Kyuhan Lee, Kijung Shin, Huawei Shen and Xueqi Cheng. "DPGS: Degree-Preserving Graph Summarization." In SDM, 2021.

7. <span id="ref7"></span> Zhang, Jiabao, Shenghua Liu, Wenjian Yu, Wenjie Feng, and Xueqi Cheng. "Eigenpulse: Detecting surges in large streaming graphs with row augmentation." In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 501-513. Springer, Cham, 2019.

8. <span id="ref8"></span> Li, Xiangfeng, Shenghua Liu, Zifeng Li, Xiaotian Han, Chuan Shi, Bryan Hooi, He Huang, and Xueqi Cheng. "Flowscope: Spotting money laundering based on graphs." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, pp. 4731-4738. 2020.

9. <span id="ref9"></span> LeFevre, Kristen, and Evimaria Terzi. "GraSS: Graph structure summarization." In Proceedings of the 2010 SIAM International Conference on Data Mining, pp. 454-465. Society for Industrial and Applied Mathematics, 2010.

10. <span id="ref10"></span> Ferraz Costa, Alceu, Yuto Yamaguchi, Agma Juci Machado Traina, Caetano Traina Jr, and Christos Faloutsos. "Rsc: Mining and modeling temporal activity in social media." In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 269-278. 2015.

11. <span id="ref11"></span> Sun, Xiaobing, Jiabao Zhang, Qiming Zhao, Shenghua Liu, Jinglei Chen, Ruoyu Zhuang, Huawei Shen, and Xueqi Cheng. "CubeFlow: Money Laundering Detection with Coupled Tensors." In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 78-90. Springer, Cham, 2021.

12. <span id="ref12"></span> Feng, Wenjie, Shenghua Liu, Danai Koutra, Huawei Shen, and Xueqi Cheng. "Specgreedy: unified dense subgraph detection." In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 181-197. Springer, Cham, 2020.

13. <span id="ref13"></span> Sun, Xiaobing, Wenjie Feng, Shenghua Liu, Yuyang Xie, Siddharth Bhatia, Bryan Hooi, Wenhan Wang, and Xueqi Cheng. "MonLAD: Money Laundering Agents Detection in Transaction Streams." arXiv preprint arXiv:2201.10051 (2022).

14. <span id="ref14"></span> Bryan Hooi, Shenghua Liu, Asim Smailagic, and Christos Faloutsos, “BEATLEX: Summarizing and Forecasting Time Series with Patterns,” The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), Skopje, Macedonia, 2017.

15. <span id="ref15"></span> Zhou, Bin, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. "BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series." In IJCAI, pp. 4433-4439. 2019.

16. <span id="ref16"></span> Liu, Shenghua, Bin Zhou, Quan Ding, Bryan Hooi, Zheng bo Zhang, Huawei Shen, and Xueqi Cheng. "Time Series Anomaly Detection with Adversarial Reconstruction Networks." IEEE Transactions on Knowledge and Data Engineering (2022).