I recently gave a talk at python web conf 2022, and one of the things I did
when I should have been working on my presentation was workig on how my
presentation looked… classic procrastination technique.
Slide One # [1]
Lets use this section to show what it looks like as I change my styles.
from markata import Markata
Markata()
markata.run()
☝ This is how my website is built
- write markdown
- build site
- publish
default # [2]
This is what the above slide looks like in lookatme.
[3]
Set focus to the most important element # [4]
The way I write my slides I want the most prominant element to be the slides
title, not the presentation title. The slides title is generally the point I
am trying to make, I will leave some supporting information if I want, but
sometimes, I just have a title.
styles:
title:
bg: default
fg: '#e1af66'
headings:
'1':
bg: default
fg: '#ff66c4,bold,italics'
prefix: ' ⇁ '
suffix: ' ↽ '
[5]
by default he prefix/suffix was a full block that just went transparant into
the slide. I thought the harpoons were fun and went with them on a whim
The box characters bother me # [6]
The box characters are fine really, but it really bothers me th...
Publishing rhythm
I use a package
eyeseast/python-frontmatter [1]{.hoverlink}
to load files with frontmatter in them. Its a handy package that allows you to
load files with structured frontmatter (yaml, json, or toml).
Install # [2]
It’s on pypi, so you can install it into your virtual environment [3] with pip.
python -m pip install python-frontmatter
🙋 What’s Frontmatter # [4]
Frontmatter is a handy way to add metadata to your plain text files. It’s
quite common to have yaml frontmatter in markdown. All of my blog posts have
yaml frontmatter to give the post metadata such as post date, tags, title, and
template. dev.to is a popular developer blogging platform that also builds all
of its posts with markdown and yaml frontmatter.
Let’s see an example # [5]
Here is the exact frontmatter for this post you are reading on my site.
---
date: 2022-03-24 03:18:48.631729
templateKey: til
title: How I load Markdown in Python
tags:
- linux
- python
---
This is where the markdown content for the post goes.
So it’s yaml # [6]
yaml is the most commmon, but
python-frontmatter [7]{.hoverlink}
also supports
Handlers [8]{.hoverlink}
for toml and json.
If you want a good set of examples of yaml
learnxi...
I like WaylonWalker’s [1] project markata [2].
A plugins all the way down static site generator written in python. Plugins all the way down means that you can completely change the behavior of how it works by swapping plugins, installing new ones, or creating your own all in python.
References:
[1]: https://github.com/WaylonWalker
[2]: https://github.com/WaylonWalker/markata
Today I was watching the python web conf 2022 and saw
@davidbujic [1] use the new Dict Union Operator
Live on stage during his Functional
Programming [2]
talk. This operator was first introduced into python 3.9 with pep584 [3].
Merge Dicts # [4]
I’ve long updated dicts through the use of unpacking. Note that the last item
always wins. It makes it pretty easy to make user overrides to default
configurations. With pep584 landing in python 3.9 we can now leverage the |
operator to achieve the same result.
default_config = {'url': 'https://example.com', 'assets_dir': 'static' }
user_config = {'url': 'https://waylonwalker.com'}
# **unpacking goes back much further than 3.9
config = {**default_config, **user_config}
print(config)
# {'url': 'https://waylonwalker.com', 'assets_dir': 'static'}
# the same can be achieved through the new to python 3.9 | operator
config = default_config | user_config
print(config)
# {'url': 'https://waylonwalker.com', 'assets_dir': 'static'}
understanding python *args and **kwargs [5]
More on unpacking in this post.
Update Dicts # [6]
With the release there is also a new update syntax |= that you can use to
update. I dont often mutate variables fo...
I love the freedom of writing in markdown. It allows me to write content from
the comfort of my editor with very little focus on page style. It turns out
that markdown is also a fantastic tool for creating slides.
Present from the terminal # [1]
I will most often just present right from the terminal using
lookatme [2]. Presenting
from the terminal lets me see the results quick right from where I am editing.
It also allows me to pop into other terminal applications quickly.
reveal.js # [3]
I sometimes also use reveal.js, but that’s for another post. It is handy that
it lives in the browser and is easier to share.
New Slides # [4]
I leverage auto slides when I write my slides in markdown. The largest
heading, usually an h2 for me, becomes the new slide marker. Otherwise my
process is not much different, It just becomes a shorter writing style.
Installation # [5]
lookatme is a python library that is available on pypi, you can install it with
the pip command.
python -m pip install lookatme
Since it’s a command line application it works great with pipx. This prevents
the need to manage virtual environments yourself or ending up with packages
clashing in your system python e...
Check out eyeseast [1] and their project python-frontmatter [2].
Parse and manage posts with YAML (or other) frontmatter
References:
[1]: https://github.com/eyeseast
[2]: https://github.com/eyeseast/python-frontmatter
When I need to read contents from a plain text file in python I find the
easiest way is to just use Pathlib.
from pathlib import Path
Path('path_to_file').read_text()
Let’s make a vim command to automatically collect all the links in these
posts at the end of each article. Regex confuses the heck out of me…
I don’t have my regex liscense, but
regex can be so darn powerful especially in an editor.
Step one # [1]
Before you run someone’s regex from the internet that you don’t fully
understand, check your git status and make sure you are all clear with
git [2] before you wreck something
Inspiration # [3]
Something that I have always appreciated form
Nick Janetakis [4] is his links section. I
often try to gather up the links at the end of my posts, but often end
up not doing it or forgetting.
Making a Links section # [5]
Searchng through the internet I was able to find an article from
Vitaly Parnas called
vim ref links [6] that did
almost exactly what I needed, except it was more complicated and made
them into ref liks.
Here is my interpretation of the code I took from Vitaly’s post. It
makes a Links section like the one at the bottom of this post.
function! MdLinks()
$norm o## Links
$norm o
g/\[[^\]]\+\]([^)]\+)/t$
silent! '^,$s/\v[^\[]*(\[[^\]]+\])\(([^)]+)\)[^\[]*/* \1(\2)/g
nohl
endfunction
command! MdLinks call MdLinks()
So far ...
I came across awesome-tuis [1] from rothgar [2], and it’s packed with great features and ideas.
List of projects that provide terminal user interfaces
References:
[1]: https://github.com/rothgar/awesome-tuis
[2]: https://github.com/rothgar
If you ever end up on a linux machine that just does not have enough ram to
suffice what you are doing and you just need to get the job done you can give
it some more swap. You can look up reccomendations for how much swap you
should have this is more about just trying to get your job done when you are
almost there, but running out of memory on the hardware you have.
make the /swap file # [1]
You can put this where you wish, for this example I am going to pop it into
/swap
sudo fallocate -l 4G /swap
sudo chmod 600 /swap
sudo mkswap /swap
sudo swapon /swap
make sure that your swap is on # [2]
You can make sure that your swap is working by using the free command, I like
using the -h flag to get human readable numbers.
❯ free -h
total used free shared buff/cache available
Mem: 15Gi 5.5Gi 4.9Gi 458Mi 5.2Gi 9.3Gi
Swap: 4.0Gi 0B 4.0Gi
Reclaim memory usage in Jupyter [3]
I also used this trick in this article to give my python process a bit more oompf and get it on home.
References:
[1]: #make-the-swap-file
[2]: #make-sure-that-your-swap-is-on
[3]: /reset-ipython/
A very common task for any script is to look for files on the system. My go to
method when globbing for files in python is to use pathlib.
Setup # [1]
I setup a directory to make some examples about globbing. Here is what the
directory looks like.
❯ tree .
.
├── content
│ ├── hello.md
│ ├── hello.py
│ ├── me.md
│ └── you.md
├── readme.md
├── README.md
├── READMES.md
└── setup.py
1 directory, 8 files
Pathlib # [2]
Pathlib is a standard library module available in all LTS versions of python at
this point.
❯ from pathlib import Path
Creating a Path instance.
# current working directory
Path()
Path.cwd()
# The users home directory
Path.home()
# Path to a directory by string
Path('/path/to/directory')
# The users ~/.config directory
Path.home() / '.config'
Globbing Examples # [3]
The path object has a glob method that allows you to glob for files with a unix
style glob pattern to search for files. Note that it gives you a generator.
This is great for many use cases, but for examples its easier to turn them to a
list to print them out.
If you need some more detail on what globbing is there is a
wikipedia [4] article
discussing it. I am just showing how to glob with pathlib...
Setting up your git [1] pager to your liking can help you navigate diffs and logs
much more efficiently. You can set it to whatever pager you like so that your
keys feel nice and smooth and your fingers know exactly what to do. You might
even gain a few extra features.
Setting the pager # [2]
You can set the pager right from your command line with the following command.
git config --global core.pager 'more'
You can also set your pager by editing your global .gitconfig file which by
default is set to ~/.gitconfig.
[core]
pager = more
Color # [3]
In my experience you need to turn colors off with nvim. bat handles them and
looks good either way, but nvim will be plain white and display the color
codes as plain text if color is on.
git config --global color.pager no
Pagers I have tried # [4]
Here are some various configs that I tried. For some reason line numbers in
bat really bothered me, but when in nvim they felt ok. I am going to try
running both of them for a few days and see which I like better. I think
having some of my nvim config could be really handy for things like yanking a
commit hash to the system clipboard without touching the mouse.
# bat
git config --global...
If you have ever mistyped a git [1] command very close to an existing one
you have likely seen this message.
❯ git chekout dev
git: 'chekout' is not a git command. See 'git --help'.
The most similar command is
checkout
Automatically run the right one # [2]
What you might not have known is that you can configure git to just run
this command for you.
# Gives you 0.1 seconds to respond
git config --global help.autocorrect 1
# Gives you 1 seconds to respond
git config --global help.autocorrect 10
# Gives you 5 seconds to respond
git config --global help.autocorrect 50
Fat Fingers Gone # [3]
Now when you typo a git command it will autmatically run after the
configured number of tenths of a second.
❯ git chkout get-error
WARNING: You called a Git command named 'chkout', which does not exist.
Continuing in 1.0 seconds, assuming that you meant 'checkout'.
M pages/blog/how-i-deploy-2021.md
M pages/hot_tips/001.md
M pages/templates/gratitude_card.html
M plugins/index.py
M plugins/publish_amp.py
M plugins/render_template_variables.py
M plugins/youtube.py
M requirements.txt
M static/index.html
Switched to branch 'get-error'
My config # [4]
I’m rocking 10 for now just to see how I ...
yq is a command line utility for parsing and querying yaml, like jq does for json.
This is for me # [1]
I love that all of these modern tools built in go and rust, just give you a
zipped up executable right from GitHub releases, but it’s not necessarily
straight forward how to install them. yq does one of the best jobs I have
seen, giving you instructions on how to get a specific version and install it.
I use a bunch of these tools, and for what its worth I trust the devs behind
them to make sure they don’t break. This so far has worked out well for me,
but if it ever doesn’t I can always pick an older version.
Just give me the latest # [2]
Since I am all trusting of them I just want the latest version. I do not want
to update a shell script with new versions, or even care about what then next
version is, I just want it. Luckily you can script the release page for the
latest version on all that I have came accross.
What is the latest # [3]
I wrote or stole, I think I wrote it, this line of bash quite awhile ago, and
it has served me well for finding the latest release for any GitHub project
using releases. Just update it with the name of the tool, org, and repo and it
wor...
Last Thursday I learned about pytest-mock at a local python meetup. The
presenter showed how he uses pytest-mock for his work, and it was kinda eye
opening. I knew what mocking was, but I had not seen it in this context.
Discovery # [1]
Watching him use pytest-mock I realized that mocking was not as hard as I had
made it out to be. You can install pytest-mock, use the mocker fixture, and
patch objects methods with what you want them to be.
install # [2]
pytest-mock is out on pypi and can be installed with pip.
python -m pip install pytest-mock
What I actually did # [3]
Sometimes I fall victim to making these posts nice and easy to follow. It
takes more steps than just pip install, you need a place to practice in a nice
sandbox. Here is how I make my sandboxes.
mkdir ~/git/learn-pytest-mock
cd ~/git/learn-pytest-mock
# well actually open a new tmux session there
echo pytest-mock > requirements.txt
# I copied in my .envrc, and ran direnv allow, which actually just made me a virtual env as follows
python3 -m venv .venv --prompt $(basename $PWD)
source .venv/bin/activate
# now install pytest-mock
pip install -r requirements.txt
# make some tests to mock
mkdir tests
nvim t...
Python 3.8 came out two and a half years ago and I have yet to really lean in
on the walrus operator. Partly because it always seemed like something kinda
silly (my use cases) to require a python version bump for, and partly because I
really didn’t understand it the best. Primarily I have wanted to use it in
comprehensions, but I did not really understand how.
Now that Python 3.6 is end of life, and most folks are using at least 3.8 it
seems time to learn and use it.
What’s a Walrus # [1]
:=
The assignment operator in python is more commonly referred to as the walrus
operator due to how := looks like a walrus. It allows you to assign and use
a variable in a single expression.
This example from the docs avoids a second call to the len function.
if (n := len(a)) > 10:
print(f"List is too long ({n} elements, expected <= 10)")
Let’s get some data # [2]
without a walrus
In this example we are going to do a dict comp to generate a map of content
from urls, only if their status code is 200. When doing this in a dictionary
comprehension we end up needing to hit the url twice for successful urls. Once
for the filter and once for the data going into the dictionary.
{
url: reque...
I’m impressed by markdown-styles [1] from mixu [2].
Markdown to static HTML [3] generator and multiple CSS themes for Markdown
References:
[1]: https://github.com/mixu/markdown-styles
[2]: https://github.com/mixu
[3]: /html/
Kedro rich is a very new and unstable (it’s good, just not ready) plugin for
kedro to make the command line prettier.
Install kedro rich # [1]
There is no pypi package yet, but it’s on github. You can pip install it with
the git [2] url.
pip install git+https://github.com/datajoely/kedro-rich
Kedro run # [3]
You can run your pipeline just as you normally would, except you get progress
bars and pretty prints.
kedro run
[4]
Kedro catalog # [5]
Listing out catalog entries from the command line now print out a nice pretty
table.
kedro catalog list
[6]
Give it a star # [7]
Go to the GitHub repo [8] and give it a
star, Joel deserves it.
References:
[1]: #install-kedro-rich
[2]: /glossary/git/
[3]: #kedro-run
[4]: https://dropper.waylonwalker.com/file/fc807030-c03e-42e7-a721-f7787589c85a.webp
[5]: #kedro-catalog
[6]: https://dropper.waylonwalker.com/file/f254966b-c8e3-479d-b9df-130dfdc4e46f.webp
[7]: #give-it-a-star
[8]: https://github.com/datajoely/kedro-rich
So worktrees, I always thought they were a big scary things. Turns out they
are much simpler than I thought.
Myth #1 # [1]
no special setup
I thought you had to be all in or worktrees or normal git [2], but not both. When
I see folks go all in on worktrees they start with a bare repo, while its true
this is the way you go all in, its not true that this is required.
Lets make a worktree # [3]
Making a worktree is as easy as making a branch. It’s actually just a branch
that lives in another place in your filesystem.
# checkout a new worktree called compare based on main in /tmp/project
git worktree add -b compare /tmp/project main
# checkout a new worktree called compare based on HEAD in /tmp/project
git worktree add -b compare /tmp/project
# checkout a worktree from an existing feature branch in /tmp/project
git worktree add /tmp/project my-existing-feature-branch
The worktree that you create is considered a linked worktree, while the
original worktree is called the main worktree
Note that I put this in my tmp directory because I don’t expect it to live very
long, my recent use case was to compare two files after a big formatting
change. You put these where you want, bu...
nicoddemus [1] has done a fantastic job with pytest-rich [2]. Highly recommend taking a look.
pytest + rich integration (proof of concept)
References:
[1]: https://github.com/nicoddemus
[2]: https://github.com/nicoddemus/pytest-rich