CS 595
Testing and Verification Techniques for Machine Learning
Spring 2019


Machine Learning (ML) techniques based on neural networks are increasingly adopted in many application domains, including safety critical ones. There is an urgent need to develop effective testing and verification approaches that can improve dependability of systems that rely on ML techniques. In this seminar, we will discuss recent research results on testing and verification of ML techniques.
Instructor: Tevfik Bultan
Meeting time: Friday, 1:00PM
Location: HFH 1132
Enrollment Code: ?
Units: This will be a 2 unit seminar

Course Work


Reading List

  1. Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, Daniel Kroening: Concolic testing for deep neural networks. ASE 2018: 109-119
  2. Mohammadhosein Hasanbeig, Alessandro Abate, Daniel Kroening: Logically-Correct Reinforcement Learning. CoRR abs/1801.08099 (2018)
  3. Youcheng Sun, Xiaowei Huang, Daniel Kroening: Testing Deep Neural Networks. CoRR abs/1803.04792 (2018)
  4. Shiqing Ma, Yingqi Liu, Wen-Chuan Lee, Xiangyu Zhang, Ananth Grama: MODE: automated neural network model debugging via state differential analysis and input selection. ESEC/SIGSOFT FSE 2018: 175-186
  5. Gagandeep Singh, Timon Gehr, Markus Püschel, Martin T. Vechev: An abstract domain for certifying neural networks. PACMPL 3(POPL): 41:1-41:30 (2019)
  6. Matthew Mirman, Timon Gehr, Martin T. Vechev: Differentiable Abstract Interpretation for Provably Robust Neural Networks. ICML 2018: 3575-3583
  7. Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, Martin T. Vechev: Fast and Effective Robustness Certification. NeurIPS 2018: 10825-10836
  8. Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, Martin T. Vechev: AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. IEEE Symposium on Security and Privacy 2018: 3-18
  9. Ehlers, R.: Formal verification of piece-wise linear feed-forward neural networks. In: D’Souza, D., Narayan Kumar, K. (eds.) ATVA 2017. LNCS, vol. 10482, pp. 269–286. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68167-2 19
  10. 5. Feinman, R., Curtin, R.R., Shintre, S, Gardner, A.B.: Detecting adversarial samples from artifacts. Technical Report (2017). arXiv:1703.00410
  11. Ian J. Goodfellow, Patrick D. McDaniel, Nicolas Papernot: Making machine learning robust against adversarial inputs. Commun. ACM 61(7): 56-66 (2018)
  12. Augustus Odena, Ian J. Goodfellow: TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing. CoRR abs/1807.10875 (2018)
  13. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. Technical Report (2014). arXiv:1412.6572
  14. Huang, X., Kwiatkowska, M., Wang, S., Wu, M.: Safety verification of deep neural networks. In: Majumdar, R., Kunˇcak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 3–29. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9 1
  15. Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill: Provably Minimally-Distorted Adversarial Examples. CoRR abs/1709.10207 (2017)
  16. Guy Katz, Clark W. Barrett, David L. Dill, Kyle Julian, Mykel J. Kochenderfer: Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. CAV (1) 2017: 97-117
  17. Daniel Selsam, Percy Liang, David L. Dill: Developing Bug-Free Machine Learning Systems With Formal Mathematics. ICML 2017: 3047-3056
  18. Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, Mykel J. Kochenderfer: Towards Proving the Adversarial Robustness of Deep Neural Networks. FVAV@iFM 2017: 19-26
  19. Nicholas Carlini, David A. Wagner: Towards Evaluating the Robustness of Neural Networks. IEEE Symposium on Security and Privacy 2017: 39-57
  20. Tommaso Dreossi, Somesh Jha, Sanjit A. Seshia: Semantic Adversarial Deep Learning. CAV (1) 2018: 3-26
  21. Samuel Yeom, Irene Giacomelli, Matt Fredrikson, Somesh Jha: Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. CSF 2018: 268-282
  22. Yizhen Wang, Somesh Jha, Kamalika Chaudhuri: Analyzing the Robustness of Nearest Neighbors to Adversarial Examples. ICML 2018: 5120-5129
  23. Nicolas Papernot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, Ananthram Swami: The Limitations of Deep Learning in Adversarial Settings. EuroS&P 2016: 372-387
  24. Barreno, M., Nelson, B., Joseph, A.D., Tygar, J.D.: The security of machine learning. Mach. Learn. 81(2), 121–148 (2010)
  25. Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I.P., Tygar, J.D.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58. ACM (2011)
  26. Luca Pulina, Armando Tacchella: Challenging SMT solvers to verify neural networks. AI Commun. 25(2): 117-135 (2012)
  27. Luca Pulina, Armando Tacchella: An Abstraction-Refinement Approach to Verification of Artificial Neural Networks. CAV 2010: 243-257
  28. Francesco Leofante, Nina Narodytska, Luca Pulina, Armando Tacchella: Automated Verification of Neural Networks: Advances, Challenges and Perspectives. CoRR abs/1805.09938 (2018)
  29. Divya Gopinath, Guy Katz, Corina S. Pasareanu, Clark Barrett: DeepSafe: A Data-Driven Approach for Assessing Robustness of Neural Networks. ATVA 2018: 3-19
  30. Tommaso Dreossi, Shromona Ghosh, Alberto L. Sangiovanni-Vincentelli, Sanjit A. Seshia: Systematic Testing of Convolutional Neural Networks for Autonomous Driving. CoRR abs/1708.03309 (2017)
  31. Tommaso Dreossi, Alexandre Donzé, Sanjit A. Seshia: Compositional Falsification of Cyber-Physical Systems with Machine Learning Components. NFM 2017: 357-372
  32. Cumhur Erkan Tuncali, James Kapinski, Hisahiro Ito, Jyotirmoy V. Deshmukh: Reasoning about safety of learning-enabled components in autonomous cyber-physical systems. DAC 2018: 30:1-30:6
  33. Cumhur Erkan Tuncali, Georgios Fainekos, Hisahiro Ito, James Kapinski: Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components. Intelligent Vehicles Symposium 2018: 1555-1562
  34. Kexin Pei, Yinzhi Cao, Junfeng Yang, Suman Jana: DeepXplore: Automated Whitebox Testing of Deep Learning Systems. SOSP 2017: 1-18
  35. Yuchi Tian, Kexin Pei, Suman Jana, Baishakhi Ray: DeepTest: automated testing of deep-neural-network-driven autonomous cars. ICSE 2018: 303-314
  36. Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, Suman Jana: Efficient Formal Safety Analysis of Neural Networks. NeurIPS 2018: 6369-6379
  37. Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, Suman Jana: Formal Security Analysis of Neural Networks using Symbolic Intervals. USENIX Security Symposium 2018: 1599-1614