Adding a --pdb flag to your applications can make them much easier for
those using it to debug your application, especially if your applicatoin
is a cli application where the user has much fewer options to start this
for themselves. To add a pdb flag --pdb to your applications you will
need to wrap your function call in a try/except, and start a post_mortem
debugger. I give credit to
this stack overflow post [1]
for helping me figure this out.
import pdb, traceback, sys
def bombs():
a = []
print(a[0])
if __name__ == "__main__":
if "--pdb" in sys.argv:
try:
bombs()
except:
extype, value, tb = sys.exc_info()
traceback.print_exc()
pdb.post_mortem(tb)
else:
bombs()
Using –pdb # [2]
python yourfile.py --pdb
[3]
References:
[1]: https://stackoverflow.com/questions/242485/starting-python-debugger-automatically-on-error
[2]: #using---pdb
[3]: https://dropper.waylonwalker.com/file/c549e928-50e1-4076-8629-fa81c3bcc1c2.webp
Publishing rhythm
Converting markdown posts to pdf on ubuntu takes a few packages from the
standard repos. I had to go through a few stack overflow posts, and
nothing seemed to have all the fonts and packages that I needed to
convert markdown, but this is what ended up working for me.
Installing all the packages # [1]
sudo apt install \
pandoc \
texlive-latex-base \
texlive-fonts-recommended \
texlive-extra-utils \
texlive-latex-extra \
texlive-xetex
Using pandoc to convert markdown to a pdf # [2]
# older versions of pandoc, I needed this one on ubuntu 18.04
pandoc pages/til/convert-markdown-pdf-linux.md -o convert-markdown-pdf.pdf --latex-engine=xelatex
# newer versions of pandoc, I needed this one on ubuntu 21.04
pandoc pages/til/convert-markdown-pdf-linux.md -o convert-markdown-pdf.pdf --pdf-engine=xelatex
[3]
Here is an image of what converting this article over to a pdf looks
like. The raw markdown is
here [4].
References:
[1]: #installing-all-the-packages
[2]: #using-pandoc-to-convert-markdown-to-a-pdf
[3]: https://dropper.waylonwalker.com/file/c367ae8a-5626-4cf6-9aec-070cbf054f46.webp
[4]: https://waylonwalker.com/convert-markdown-pdf-linux.md
Python comes with an enum module for creating enums. You can make your
own enum by inheriting importing and inheriting from Enum.
from enum import Enum
class LifeCycle(Enum):
configure = 1
glob = 2
pre_render = 3
render = 4
post_render = 5
save = 6
auto incrementing # [1]
Enum values can be auto incremented by importing auto, and calling
auto() as their value.
from enum import Enum, auto
class LifeCycle(Enum):
configure = auto()
glob = auto()
pre_render = auto()
render = auto()
post_render = auto()
save = auto()
using the enum # [2]
Enum’s are accessed directy under the class itself, and have primarily
two methods underneath each thing you make, .name and .value.
Lifecycle.glob
Lifecycle.glob.value
Lifecycle.glob.name
[3]
References:
[1]: #auto-incrementing
[2]: #using-the-enum
[3]: https://dropper.waylonwalker.com/file/8a171ef2-ec49-4e5f-a779-87ada00f2066.webp
I recently paired up with another dev running windows with Ubuntu running in
wsl, and we had a bit of a stuggle to get our project off the ground because
they were missing com system dependencies to get going.
Straight in the terminal # [1]
Open up a terminal and get your required system dependencies using the apt
package manager and the standard ubuntu repos.
sudo apt update
sudo apt upgrade
sudo apt install \
python3-dev \
python3-pip \
python3-venv \
python3-virtualenv
pip install pipx
Using an Ansible-Playbook # [2]
I like running things like this through an ansible-playbook as it give me some
extra control and repeatability next time I have a new machine to setup.
- hosts: localhost
gather_facts: true
become: true
become_user: "{{ lookup('env', 'USER') }}"
pre_tasks:
- name: update repositories
apt: update_cache=yes
become_user: root
changed_when: False
vars:
user: "{{ ansible_user_id }}"
tasks:
- name: Install System Packages 1 (terminal)
become_user: root
apt:
name:
- build-essential
- python3-dev
- python3-pip
- python3-venv
- python3-virtualenv
- name: check is pipx installed
shell: command -v pipx
register: pipx_exists
ignore_errors: y...
Stow is an incredible way to manage your dotfiles. It works by managing
symlinks between your dotfiles directory and the rest of the system. You can
then make your dotfiles directory a git [1] repo and have it version controlled. In
my honest opinion, when I was trying to get started the docs straight into deep
detail of things I frankly don’t really care about and jumped right over how to
use it.
When using stow its easiest to keep your dotfiles directory (you may name it
what you want) in your home directory, with application directories inside of
it.
Then each application directory should reflet the same diretory structure as you
want in your home directory.
zsh # [2]
Here is a simple example with my zshrc.
mkdir ~/dotfiles
cd ~/dotfiles
mkdir zsh
mv ~/.zshrc zsh
stow --simulate zsh
You can pass in the –simulate if you wish, it will tell you if there are going
to be any more errors or not, but it wont give much more than that.
WARNING: in simulation mode so not modifying filesystem.
Once your ready you can stow your zsh application.
stow zsh
nvim # [3]
A slightly more complicated example is neovim since its diretory structure does
not put configuration files directl...
Check out sharkdp [1] and their project pastel [2].
A command-line tool to generate, analyze, convert and manipulate colors
References:
[1]: https://github.com/sharkdp
[2]: https://github.com/sharkdp/pastel
If you’re into interesting projects, don’t miss out on asdf [1], created by asdf-vm [2].
Extendable version manager with support for Ruby, Node.js, Elixir, Erlang & more
References:
[1]: https://github.com/asdf-vm/asdf
[2]: https://github.com/asdf-vm
The copier answers file is a key component to making your templates
re-runnable. Let’s look at the example for my setup.py.
❯ tree ~/.copier-templates/setup.py
/home/walkers/.copier-templates/setup.py
├── [[ _copier_conf.answers_file ]].tmpl
├── copier.yml
├── setup.cfg
└── setup.py.tmpl
0 directories, 4 files
Inside of my [[ _copier_conf.answers_file ]].tmpl file is this, a
message not to muck around with it, and the ansers in yaml form. The
first line is just a helper for the blog post.
# ~/.copier-templates/setup.py/\[\[\ _copier_conf.answers_file\ \]\].tmpl
# Changes here will be overwritten by Copier; NEVER EDIT MANUALLY
[[_copier_answers|to_nice_yaml]]
Inside my copier.yml I have setup my _answers_file to point to a special
file. This is because this is not a whole projet template, but one just
for a single file.
# copier.yml
# ...
_answers_file: .setup-py-copier-answers.yml
Once I change the _answers_file I was incredibly stuck
Run it # [1]
I’m making a library of personal copier templates in my
~/.copier-templates directory and I am going to run it from there.
copier copy ~/.copier-templates/setup.py
Results # [2]
After rendering the template we have the followi...
Once you have made your sick looking cli apps with rich, eventually you are
going to want to add some keybindings to them. Currently Textual, also written
by [1]@willmcgugan [2], does this extremely well.
Fair Warning it is in super beta mode and expected to change a bunch. So take
it easy with hopping on the train so fast.
Get the things # [3]
Install them from the command line.
pip install textual
pip install rich
Import make a .py file and import them in it.
from textual.app import App
from textual.widget import Widget
from rich.panel import Panel
Make what you have a widget # [4]
If you return your rich renderable out of class that inherits from
textual.widget.Widget, you can then dock this inside of an app class
inheriting from textual.app.App.
class MyWidget(Widget):
def render(self):
my_renderable = Panel("press q to quit")
return my_renderable
class MyApp(App):
async def on_mount(self) -> None:
await self.view.dock(MyWidget(), edge="top")
await self.bind("q", "quit")
run it # [5]
You’ve made a TUI (text user interface). Run the classmethod run to display
the it in its full screen glory.
MyApp.run(log="textual.log")
Final result # [6]
At this point It prob...
I’ve been stuck many times looking at a vim buffer with little question
marks at the beginning of each line and trying to get rid of them. for
so long I didn’t know what they were so trying to get rid of them was
impossible.
[1]
It turns out they are tabs, and you can get rid of the little leading
question marks with this substitution command.
:%s/\t/ /g
References:
[1]: https://dropper.waylonwalker.com/file/cd79ed2e-be03-4173-a9fd-4f67c04b9b70.webp
tmux popups can be sized how you like based on the % width of the
terminal on creation by using the flags (h, w, x, y) for height, width,
and position.
# normal popup
tmux popup figlet "Hello"
# fullscreen popup
tmux popup -h 100% -w 100% figlet "Hello"
# 75% centered popup
tmux popup -h 100% -w 75% figlet "Hello"
# 75% popup on left side
tmux popup -h 100% -w 75% -x 0% figlet "Hello"
Sorry, your browser doesn't support embedded videos.
example running these commands
I was completely stuck for awhile. copier was not replacing my template
variables. I found out that adding all these _endops fixed it. Now
It will support all of these types of variable wrappers
# copier.yml
_templates_suffix: .jinja
_envops:
block_end_string: "%}"
block_start_string: "{%"
comment_end_string: "#}"
comment_start_string: "{#"
keep_trailing_newline: true
variable_end_string: "}}"
variable_start_string: "{{"
!RTFM: Later I read the docs and realized that copier defaults to using [[
and ]] for its templates unlike other tools like cookiecutter.
I’ve been looking for a templating tool for awhile that works well with
single files. My go to templating tool cookiecutter does not work for
single files, it needs to put files into a directory underneath of it.
template variables # [1]
By default copier uses double square brackets for its variables.
variables in files, directory_names, or file_names will be substituted
for their value once you render them.
# hello-py/hello.py.tmpl
print('hello-[[name]]')
note! by default copier will not inject variables into your
template-strings unless you use a .tmpl suffix.
Before running copier we need to tell copier what variables to ask for,
we do this with a copier.yml file.
# copier.yml
name:
default: my_name
type: str
help: What is your name
installing copier # [2]
I prefer to install cli tools that I need globally with pipx, this
always gives me access to the tool without worrying about dependency
conflicts, bloating my system site-packages, or managing a separate
virtual environment [3] for it myself.
pipx install copier
running copier # [4]
When running copier copy we pass in the directory of the template, and
the directory that we want to render the template into.
cop...
I just installed a brand new Ubuntu 21.10 Impish Indri, and wanted a
kedro project to play with so I did what any good kedroid would do, I
went to my command line and ran
pipx run kedro new --starter spaceflights
But what I got back was not what I expected!
Fatal error from pip prevented installation. Full pip output in file:
/home/walkers/.local/pipx/logs/cmd_2022-01-01_20.42.16_pip_errors.log
Some possibly relevant errors from pip install:
ERROR: Could not find a version that satisfies the requirement kedro (from versions: none)
ERROR: No matching distribution found for kedro
Error installing kedro.
This is weird, why cant I run kedro new with pipx? Lets try pip.
pip install kedro
Same issue.
ERROR: Could not find a version that satisfies the requirement kedro (from versions: none)
ERROR: No matching distribution found for kedro
What is Kedro [1]
Curious what kedro is? Check out this article.
What’s up # [2]
wrong python version
The issue is that kedro only runs on up to python 3.8, and on Ubuntu
21.10 when you apt install python3 you get python 3.9 and the
standard repos don’t have an old enough version to run kedro.
How to fix this? # [3]
Theres a couple of wa...
Pluggy makes it so easy to allow users to modify the behavior of a framework
without thier specific feature needing to be implemented in the framework
itself.
I’ve really been loving the workflow of frameworks built with pluggy. The first
one that many python devs have experience with is pytest. I’ve never created a
pytest plugin, and honestly at the time I looked into how they were made was a
long time ago and it went over my head. I use a data pipelining framework
called kedro, and have build many plugins for it.
Making a plugin # [1]
super easy to do
As long as the framework document the hooks that are available and what it
passes to them it’s so easy to make a plugin. Its just importing the
hook_impl, making a class with a function that represents one of the hooks,
and decorating it.
from framework import hook_impl
class LowerHook:
@hook_impl
def start(pluggy_example):
pluggy_example.message = pluggy_example.message.lower()
installing pluggy # [2]
Installing pluggy is just like most python applications, install python, make
your virtual environment [3], and pip install it.
pip install pluggy
Making a plugin driven framework # [4]
much less easy
At the time I sta...
One of the most useful skills you can acquire to make you faster at
almost any job that uses a computer is getting good at finding text in
your current working diretory and identifying the files that its in. I
often use the silver searcher ag or ripgrep rg to find files in
large directories quickly. Both have a sane set of defaults that ignore
hidden and gitignored files, but getting them to list only the filenames
and not the matched was not trivial to me.
I’ve searched throught he help/man pages many times looking for these
flags and they always seem to evade me.
ag # [1]
Passing the flag -l to ag will get it to list only the filepath, and
not the match. Here I gave it a --md as well to only return markdown
filetypes. ag supports a number of filetypes in a very similar way.
ag nvim --md -l
rg # [2]
Giving rg the --files-with-matches flag will yield you a similar set
of results, giving only the filepaths themselves and not the match
statement. Also passing in the -g "*.md" will similarly yield only
results from markdown files.
rg --files-with-matches you -g "*.md"
References:
[1]: #ag
[2]: #rg
pyenv provides an easy way to install almost any version of python from
a large list of distributions. I have simply been using the version of
python from the os package manager for awhile, but recently I bumped my
home system to Ubuntu 21.10 impish, and it is only 3.9+ while the
libraries I needed were only compatable with up to 3.8.
I needed to install an older version of python on ubuntu
I’ve been wanting to check out pyenv for awhile now, but without a
burning need to do so.
installing # [1]
Based on the Readme it looked like I needed to install using homebrew,so this
is what I did, but I later realized that there is a pyenv-installer repo that
may have saved me this need.
Installing Homebrew on Linux [2]
List out install candidates # [3]
You can list all of the available versions to install with
pyenv install --list. It does reccomend updating pyenv if you suspect
that it is missing one. At the time of writing this comes out to 532
different versions!
pyenv install --list
Let’s install the latest 3.8 patch # [4]
Installing a version is as easy as pyenv install 3.8.12. This will
install it, but not make it active anywhere.
pyenv install 3.8.12
let’s use python 3.8...
I came across outputformat [1] from delestro [2], and it’s packed with great features and ideas.
Python library to decorate and beautify strings
References:
[1]: https://github.com/delestro/outputformat
[2]: https://github.com/delestro
Installing brew on linux proved quite easy and got pyenv running for me
within 4 commands.
I had never used homebrew before, honestly I thought it was a mac only
thing for years. Today I wanted to try out pyenv, and the reccommended
way to install was using homebrew. I am not yet sure if I want either
in my normal workflow, so for now I am just going to pop open a new
terminal and install homebrew and see how it goes.
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /home/walkers/.zprofile
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
That was it, now homebrew is working. Starting a new shell and running
the command to install pyenv worked.
brew install pyenv
Links # [1]
- homebrew [2]
References:
[1]: #links
[2]: https://brew.sh/
pyenv [1] has done a fantastic job with pyenv [2]. Highly recommend taking a look.
Simple Python version management
References:
[1]: https://github.com/pyenv
[2]: https://github.com/pyenv/pyenv