One thing I’m thinking is Python’s types aren’t really checked at execution time. This technique should probably go a long way in helping make sure new code which is carefully typed is safer, but it may be hard to apply to an existing/older project.
One other thing I want to note (and I know it is beside the point of the article, which is demonstrating types as a tool for enforcing a workflow :)) is tools like the subprocess module lets you explicitly pass a list of arguments instead a string of arguments. This skips the shell altogether, and makes sure that the data passed in is passed literally to the program as an argument.
One thing I’m thinking is Python’s types aren’t really checked at execution time.
Many natively-statically-typed languages also don’t do or at least significantly scale back runtime checks. In fact, often they argue that they can offer performance benefits precisely because the overhead of checking those things happens in advance of running the program.
Does this mean you could write some Python code that does a bad thing, refuse to use the static checker (or ignore its results), and then run the code to do the bad thing? Sure, but nobody ever promised otherwise, and someone who’s determined to bypass rules and tools and systems put in place for safety in a software project can basically always do a lot of damage. The article’s technique of using type hints to set expectations, coupled with something like a CI gate that breaks the build on a type-check failure, is still an improvement for someone who’s worried about misuse of an API like the given example.
mypyc (mypy compiler) for python crosses that line: compiled code will raise TypeError at runtime if things turn out to be different from what the type checker + compiler expected
There’s also typeguard that you can use to check types at runtime with python. I think this can be especially useful to add as part of the test suite where the overhead (and failure) are easier to manage.
Typing is very useful to express semantic and bound context. By typing security semantic, a reviewer only need to check the implementation and the semantic of the typing. Linting then do most of the context-heavy work by ensuring the use of these explicit patterns. Developers then don’t have to remember all the rules and reviewing becomes much faster when you know it’s not possible to misuse the API.
If the semantic and its implementation is correct and the linter is working properly, then there’s no reason the runtime behavior would lead to unexpected behavior.
As for typing existing projects, I think this is where it shines the most! Security reviewers can type any unsafe function and rely on the linter to trace back values and then explicitly whitelist quoting function until all errors are silenced.
This is my first written article ever published, I welcome any feedback, whether about writing style or the content itself.
Pretty good, thank you for sharing it!
One thing I’m thinking is Python’s types aren’t really checked at execution time. This technique should probably go a long way in helping make sure new code which is carefully typed is safer, but it may be hard to apply to an existing/older project.
One other thing I want to note (and I know it is beside the point of the article, which is demonstrating types as a tool for enforcing a workflow :)) is tools like the subprocess module lets you explicitly pass a list of arguments instead a string of arguments. This skips the shell altogether, and makes sure that the data passed in is passed literally to the program as an argument.
Many natively-statically-typed languages also don’t do or at least significantly scale back runtime checks. In fact, often they argue that they can offer performance benefits precisely because the overhead of checking those things happens in advance of running the program.
Does this mean you could write some Python code that does a bad thing, refuse to use the static checker (or ignore its results), and then run the code to do the bad thing? Sure, but nobody ever promised otherwise, and someone who’s determined to bypass rules and tools and systems put in place for safety in a software project can basically always do a lot of damage. The article’s technique of using type hints to set expectations, coupled with something like a CI gate that breaks the build on a type-check failure, is still an improvement for someone who’s worried about misuse of an API like the given example.
mypyc (mypy compiler) for python crosses that line: compiled code will raise
TypeErrorat runtime if things turn out to be different from what the type checker + compiler expectedThere’s also typeguard that you can use to check types at runtime with
python. I think this can be especially useful to add as part of the test suite where the overhead (and failure) are easier to manage.Also Pydantic which gives you runtime checks for types. Its v2 is being written in Rust to make that faster/better.
Typing is very useful to express semantic and bound context. By typing security semantic, a reviewer only need to check the implementation and the semantic of the typing. Linting then do most of the context-heavy work by ensuring the use of these explicit patterns. Developers then don’t have to remember all the rules and reviewing becomes much faster when you know it’s not possible to misuse the API.
If the semantic and its implementation is correct and the linter is working properly, then there’s no reason the runtime behavior would lead to unexpected behavior.
As for typing existing projects, I think this is where it shines the most! Security reviewers can type any unsafe function and rely on the linter to trace back values and then explicitly whitelist quoting function until all errors are silenced.
There’s a library for that: https://github.com/beartype/beartype
Thanks for sharing, I didn’t know about this and this look great!
This is a very good use of the type system!
Of course, the deeper fix is to just not have os.system, since it’s an attractive nuisance. But unfortunately it exists and is widely used.