Published

The next version of markata will be around a full second faster at building it’s docs, that’s a 30% bump in performance at the current state. This performance will come when virtual environments are stored in the same directory as the source code.

What happened?? #

I was looking through my profiler for some unexpected performance hits, and noticed that the docs plugin was taking nearly a full second (sometimes more), just to run glob.

    |  |- 1.068 glob  markata/plugins/docs.py:40
    |  |  |- 0.838 <listcomp>  markata/plugins/docs.py:82
    |  |  |  `- 0.817 PathSpec.match_file  pathspec/pathspec.py:165
    |  |  |        [14 frames hidden]  pathspec, <built-in>, <string>

Python scandir ignores hidden directories #

I started looking for different solutions and what I found was that I was hitting pathspec with way more files than I needed to.

len(list(Path().glob("**/*.py")))
# 6444
len([Path(f) for f in glob.glob("**/*.py", recursive=True)])
# 110

After digging into the docs I found that glob.glob uses os.scandir which ignores ‘.’ and ‘..’ directories while Path.glob does not.

https://docs.python.org/3/library/os.html#os.scandir

results? #

Now glob.py from the docs plugin does not even show up in the profiler.

I opened up ipython and saw the following results. For some reason as I hit docs.glob it was only hitting 488 ms from ipython, but it was still a massive improvement over the original.

%timeit docs.glob(m)
# 488 ms ± 3.05 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit docs.glob(m)
# 9.37 ms ± 90.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

AUR.">paru is an aur helper that allows you to use a package manager to install packages from the aur.

What’s the Aur #

The Aur is a set of community managed packages that can be installed on arch based distros.

Why a helper? #

paru just makes it easy, no clone and run makepkg. You can do everything paru can do using the built in pacman installer.

Manual Install from the Aur #

You will need to manually instal pacman from the aur in order to get started.

sudo pacman -S --needed base-devel
git clone https://aur.archlinux.org/paru.git
cd paru
makepkg -si

Installing packages with paru #

Once setup you are ready to install packages from the AUR just like the core repos.

# you can update your system using paru
paru -Syu

# you can install packages from the AUR
paru -S tailscale
paru -S prismlauncher

# even core repo packages can be installed
paru -S docker

Paru in Docker #

Here is a snippet from my devtainer dockerfile. Where I use paru to install packages from the AUR inside of a dockerfile.

FROM archlinux

RUN echo '[multilib]' >> /etc/pacman.conf && \
    echo 'Include = /etc/pacman.d/mirrorlist' >> /etc/pacman.conf && \
    pacman --noconfirm -Syyu && \
    pacman --noconfirm -S base-devel git && \
    groupadd --gid 1000 devtainer && \
    useradd --uid 1000 --gid 1000 -m -r -s /bin/bash devtainer && \
    passwd -d devtainer && \
    echo 'devtainer ALL=(ALL) ALL' > /etc/sudoers.d/devtainer && \
    mkdir -p /home/devtainer/.gnupg && \
    echo 'standard-resolver' > /home/devtainer/.gnupg/dirmngr.conf && \
    chown -R devtainer:devtainer /home/devtainer && \
    mkdir /build && \
    chown -R devtainer:devtainer /build && \
    cd /build && \
    sudo -u devtainer git clone --depth 1 https://aur.archlinux.org/paru.git && \
    cd paru && \
    sudo -u devtainer makepkg --noconfirm -si && \
    sed -i 's/#RemoveMake/RemoveMake/g' /etc/paru.conf && \
    pacman -Qtdq | xargs -r pacman --noconfirm -Rcns && \
    rm -rf /home/devtainer/.cache && \
    rm -rf /build

USER devtainer
RUN sudo -u devtainer paru --noconfirm --skipreview --useask -S \
    bat \
    cargo \
    direnv \
    dua-cli \
    dust \
    fd

Final Thoughts #

There are other options out there, paru seemed to be the most supported at the time I started using arch and there has been no other reason for me to change it. It’s treated me well for nearly a year now.