Linting is the first step into ensuring safer, cleaner and more maintainable code. Compared to testing, it requires much less effort to set up and once it’s set up, you do not need to update it often.
In this article, I will share my approach to setup linting with flake8 for a Django project and how to integrate into your workflow using continuous integration.
What is linting and why should you care about it?
If you are new to programming or to linting, you might be asking yourself wether you should do this as a solo developer. My opinion on this is yes.
Linting provides many benefits, but essentially it increases the readability of your code. It is not only about small benefits, like using either
'' but not interchangeably, which do not seem like huge improvements, but also about making sure you space functions apart from each other in a consistent manner, which makes it much easier to find what you are looking for.
Even if you are developing alone linting is useful, as readability also matters if you haven’t looked at a file for a while.
Setting up your project for linting
First of all, we will want to create a new branch for installing the linter and using it for the first time. We will be changing a lot of files, depending on how consistent you have been coding and you might accidentally make some changes that break your code. Because of this reason, it is also preferred to be able to test your code reliably after you have set up a linter, but not required.
pylint vs. flake8
For this post, I have chosen to use flake8, as it was easier to set up than pylint, but if you are searching for a good comparison, I would recommend you check out this detailed answer on reddit.
a note on virtual environments
You will need to install flake8 on the python version your code is written in. For this and other reasons I suggest setting up a virtual environment for your project if you haven’t already, or if you don’t know what they are reading this article in the python documentation as they are a fundamental part of using python
Setting up the project
With this out of the way, we can set up our project to use flake8. Run the command
pip install flake8 flake8-django to install flake8 and the flake8-django plugin which also lints your project for Django specific best practices.
What are flake8 plugins? According to the docs:
Flake8 is useful on its own but a lot of Flake8’s popularity is due to its extensibility. Our community has developed plugins that augment Flake8’s behaviour. Most of these plugins are uploaded to PyPI. The developers of these plugins often have some style they wish to enforce.
The configuration file
The configuration file is saved at the root of your project as a
tox.ini file and allows you to customise how your linter works without needing to pass a lot of
--flags in the command-line. The two settings that I find most interesting are
exclude allows you to tell flake8 to skip scanning specific files and directories. This is great for optimising performance, as you know there aren’t any relevant files in your
.git directory and as you do not need to touch files automatically created by Django like
ignore allows you to ignore specific warnings on a project-wide level. Why would you do this? The greatest benefit a linter provides is that it makes your code style uniform. You should still be able to decide on your code style. Sure it is great to follow best practices, but you might wish to ignore some warnings like
E501: line too long for now and that shouldn’t prevent you from linting your code. I will explain how you can ignore these warnings not on a project-wide level but just for specific lines. (Which should be your first move, before you disable them entirely)
For my Django project, I use the following configuration to exclude files created by Django that I do not touch (it needs to start with
# your configuration file needs to begin with this line [flake8] # there's nothing interesting in these files exclude = .git, manage.py, lokalshoppen/asgi.py, lokalshoppen/wsgi.py, */migrations, # unused, will be removed api/serializers.py, # sometimes lines should be too long # for example with urls, etc... ignore = E501
Linting your code for the first time
Linting your code for the first time will probably use a lot of output. This is completely normal, because that shows the great effect a linter will have. But to not be overwhelmed I would suggest going about it in small steps.
Start by linting the folder containing
settings.py by running
flake8 yourfoldername you will see a bunch of warnings, like these:
casual.py:8:69: E999 SyntaxError: invalid syntax casual.py:10:3: E111 indentation is not a multiple of four casual.py:11:7: E111 indentation is not a multiple of four casual.py:12:11: E111 indentation is not a multiple of four casual.py:13:7: E111 indentation is not a multiple of four casual.py:14:11: E111 indentation is not a multiple of four casual.py:15:11: E111 indentation is not a multiple of four casual.py:15:18: E225 missing whitespace around operator casual.py:17:3: E111 indentation is not a multiple of four casual.py:18:7: E111 indentation is not a multiple of four casual.py:19:11: E111 indentation is not a multiple of four casual.py:20:7: E111 indentation is not a multiple of four casual.py:21:11: E111 indentation is not a multiple of four casual.py:22:11: E111 indentation is not a multiple of four casual.py:24:15: W292 no newline at end of file
How should you handle them? I suggest going about them in three steps:
- Navigate to the position in your code as indicated on the left hand side in the format
- Do you understand the warning? Then fix it and move to the next warning.
- Do you not understand the warning? Search for the error code on flake8rules.com to find out more about it (you can find details about the Django specific warnings on it’s GitHub page
- Finally rerun flake8 on the file to see if you have fixed all warnings. If not correct the remaining ones
Congratulations you have successfully linted the first module of your project, let’s continue through the rest of the project with this strategy:
- Pick the first Django App in your file explorer and lint it as described above
- Do you see a lot of output? Update each file of the app one by one to not be overwhelmed using the strategy explained above
- Finally, lint the whole app with
flake8 yourdjangoappto find any warnings you missed
Awesome! Now you have linted and improved your entire project step by step and are now ready to setup continuous integration (if you’d like to). Now is a good time to make a commit if you haven’t already been doing so.
Integrate flake8 into your continuous integration tool
Continuous Integration is an automation technique used to automatically check new commits, or pull requests to remote repositories to test them and deploy them to production servers. This is optional, but there really isn’t any reason to not do it as it greatly improves your workflow (who wants to be running
flake8 after every commit manually?).
I use Travis CI as it is free with the GitHub Student Developer Pack. There is a wide variety of CI tools with all sorts of features, which I will not get into in this post. Would you like to explore other options? Ask your favourite search engine or read more about which tools there are and what CI is in the amazing and free book “The Hitchhiker’s Guide to Python” The process is similar for all tools, but the configuration file is different depending on the tool you use. In the following I am assuming you are using Travis CI and have set up an account
Travis is configured via a
.travis.yml file which also sits at the root of your project. There’s a lot it can do and I encourage you to open their documentation in a new tab to read later.
Let’s go through the configuration file line by line and see what it does:
language: python python: - "X.Y" install: - pip install -r requirements.txt script: - flake8
language: python is self-explanatory
python you declare the python version you are using in the project.
install: step of your pipeline, You specify how Travis should find and install the dependencies of your project (e.g. flake8, Django, etc.). It is the easiest way to save them in a
requirements.txt file, as pip can easily install them from there.
You can generate a
requirements.txt file by running
pip freeze > requirements.txt (you should run this every time you run
script: step you put all the scripts you want to check wether the commit is valid or not. These are usually linters and tests. We’re just running
flake8 in there, which will run flake8 through your whole repository.
After you have activated Travis for your repository, you can commit and push this file. If you haven’t missed any warnings, the job should complete successfully. If you did, you will get an email and a log to see where you messed up.