Automated REST API Endpoint Identification for Security Testing at Scale: How Machine Learning Accelerates Security Testing presented at BlackHatAsia2019 2019

by Jay Chen, Azzedine Benameur, Lei Ding, Jeffrey Jacob,

Summary : Unlike traditional web applications where a web crawler is used to discover various urls, REST API endpoints can be exposed in various formats and many REST services do not provide specifications. Attackers can tamper with any part of an API request, including the url or query string to try to bypass the backend security mechanisms. Thus, it is difficult for web application scanners to identify and test APIs for vulnerabilities. Moreover, current API endpoints and parameters are identified mainly from the API documentation.In this talk, we present our approach to automatically discover and assess the security posture of APIs by leveraging machine learning, fuzzy matching, and natural language processing (NLP) techniques. We show how to automatically identify undocumented or hidden API endpoints that attackers can exploit. Our approach significantly reduces the number of probing and test times regardless of the specification of API description languages. Our tool can identify API endpoints without requiring the API documents.We will demonstrate how machine learning techniques can be used to accelerate API endpoint identification. Our approach is able to reduce the search space in terms of the number of uris. The results and open source machine learning tools we used will also be presented.