You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: security/limitations_of_AI.md
+7-2Lines changed: 7 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,9 @@
1
-
# Limitations of AI and LLMs in Python SAST
1
+
# Limitations of AI and LLMs for Python SAST tools
2
2
3
3
4
4
ML/AI solutions are not applicable to every problem. Validating Python code on security weakness is a area where using ML/AI technology should only be applied with care.
5
5
6
-
Many SAST scanners for Python code claim perfect results with the use of AI technology. Often marketing, but recent years there is an explosion of:
6
+
Many SAST (Static Application Security Testing) scanners for Python code claim perfect results with the use of AI technology. Often marketing, but recent years there is an explosion of:
7
7
- SAST scanners created with AI vibe-coding tools and
8
8
- SAST scanners that use AI agents for scanning your code.
9
9
@@ -35,6 +35,10 @@ Disadvantages of AI powered SAST scanners:
35
35
36
36
- Unlike traditional software governed by explicit, deterministic logic, **AI/ML models are probabilistic**. This means that even with identical input code, outcomes can vary due to stochastic (random) processes during training, parallel processing on hardware, and the use of 'temperature' or 'seeds' in generative outputs.
37
37
38
+
+++
39
+
- Because Python is an interpreted, late-binding language, static analysis is already difficult. **LLMs add a layer of "guessing"** to a problem that usually requires strict mathematical proof.
40
+
41
+
38
42
39
43
:::{danger}
40
44
So DO NOT rely on SAST scanners that are powered by AI-agents / LLM systems to solve your cyber security problems! AI agents are still struggling to write secure code.
@@ -46,6 +50,7 @@ So DO NOT rely on SAST scanners that are powered by AI-agents / LLM systems to s
46
50
AI solutions that are built upon LLMs for cyber security problems are still far from mature. HIDS systems (Host Intrusion Detection Systems) have a long history of applying ML technologies as well as spam-filters. Creating security products that ‘learns’ from patterns is not new for security. AI/ML technologies have been applied for many years for HIDS systems and spam-filters. Applying AI for cyber security has been done for many years with variable success.
47
51
48
52
53
+
49
54
AI/ML technologies can and do help with cyber security, especially with DAST (Dynamic Application Security Testing) and creating better fuzzers. Cyber security professionals should be conservative with adopting new IT hypes for security testing tools. IT hypes like AI-agents and LLMs are not the holy grail for solving our cyber security problems. This is because in the end you always pay more for cyber security solutions, but the risks still remain. Cyber security is not a product, but a process.
50
55
51
56
[Python Code Audit](https://codeaudit.nocomplexity.com), a modern Python-specific SAST scanner, uses AI/ML in an ethical and secure way. See the Python Code Audit [documentation](https://nocomplexity.com/documents/codeaudit/architecture.html#use-of-ai) for a deep dive into how AI/ML is used within this tool.
0 commit comments