Published

Kedro is an unopinionated Data Engineering framework that comes with a somewhat opinionated template. It gives the user a way to build pipelines that automatically take care of io through the use of abstract DataSets that the user specifies through Catalog entries. These Catalog entries are loaded, ran through a function, and saved by Nodes. The order that these Nodes are executed are determined by the Pipeline, which is a DAG. It’s the runner’s job to manage the execution of the Nodes. https://youtu.be/Wf4rnFsaFFU --- What is Kedro [1] This is an updated version of my original what-is-kedro article --- Hot Take # [2] If you are doing a series of operations to data with python, especially if you are using something as supported as pandas, you should be using a framework that gives you a pipeline as a DAG and abstracts io. Orchestrators # [3] Like I said, kedro is unopinionated it does determine where or how your data should be ran. The kedro team does support the following ...

https://youtu.be/bw5_FWDVRpU Ubuntu # [1] I recently switched over to using Ubuntu, it works well pretty much out of the box for me. I am using gnome with a dark theme. Gnome Terminal # [2] I am still using the built in default gnome terminal, it just works. It does all the things that I need it to do. It supports transparency renders my fonts and allows me to highlight things well. - One Dark Theme dotfiles # [3] You can find my dotfiles [4] on github. Feel free to read through and take anything that you find useful. I would encourage you not to steal them, but to integrate the parts that you want into your own dotfiles. dotfiles are a very personal thing. They are an extension of ones fingertips designed for how you think and type. zsh # [5] I use zsh as my default shell. I like to use it as my interactive shell. It works, and does a bit better with things like tab completion out of the box. starship # [6] I use the starship prompt for my shell. It works well out of the...

https://youtu.be/LLk94fKpGg4 As we push the limits of tmux further and further you are bound to end up in a situation where you are mashing down a hotkey and it’s just not doing what you want it to do, and you have no idea why. show-messages is a tmux command that can be used to show what tmux is actually doing behind the scenes. This might highlight any hot key conflicts you might have in your ~/.tmux.conf. man page for show-messages # [1] In case you wnat more information about show-messages, here is the man page. show-messages [-JT] [-t target-client] (alias: showmsgs) Show server messages or information. Messages are stored, up to a maximum of the limit set by the message-limit server option. -J and -T show debugging information about jobs and terminals. How I navigate tmux in 2021 [2] for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist [3] to see all of the videos in this series. References: [1]: #man-...

https://youtu.be/nT0FA1RNZEs Now your creating, jumping, and killing sessions like a boss. You are slicing through projects with ease, let me show you one more thing that can be the cream on top of this silky smooth setup we have been working towards. Chris Toomey’s [1] Tmux Course # [2] This script is simply my fork of Chris Toomey’s tat script straight out of his course. It helps us create or jump to project specific sessions with ease. a directory of projects # [3] My version of the ta script will let you pass it a directory, and it will give you a fuzzy popup. ta ~/git setting up a keybinding # [4] bind C-g display-popup -E "ta ~/git" [5] default layout # [6] By default I have my projects open with a vertical split, vim is on top, with my file finder open and the lower split is set to just my terminal. This is what I do 90% of the time that I open a project anyways. [7] More projects # [8] I also have a directory setup that is a symlink-gallery of all of my projec...

https://youtu.be/utfLA6L8o5s You’ve got some long running tasks, and you’re trying to stay productive and knock tasks off that board, but you keep finding that your processes finish and you stay on other tasks for longer than you should. You were in the flow and just did not check back in on it. With display-message you can send yourself a notification when that long running task is complete. from the man page # [1] Here is a snippet of display-message from the tmux man page. I rarely need to do anything other than just display message, but there are other flags for it. display-message [-aINpv] [-c target-client] [-d delay] [-t target-pane] [message] (alias: display) Display a message. If -p is given, the output is printed to stdout, otherwise it is displayed in the target-client status line for up to delay milliseconds. If delay is not given, the message-time option is used; a delay of zero waits for a key press. ‘N’ ignores key presses and closes only after the delay e...

https://youtu.be/QWPyYx54JbE Now it’s time to switch gears, we are onto a different part of our day and there are just too many sessions running and we need to clean up shop. kill-server # [1] One viable option is to nuke the whole dang thing. I actually do this more than you might think. tmux kill-server save and commit your work diligently before kill-server kill-session # [2] A more reasonable option might be to kill a single session. # kills the current session tmux kill-session # kills the session named scratch tmux kill-session -t scratch choose-tree # [3] Killing sessions one by one from the command line can be a bit tedious, and involve more keystrokes than necessary. Another option built right into tmux is choose-tree. By default choose-tree is bound to prefix+s, that’s pressing control+b then s. Once you are in choose-tree, you can navigate around with your configured navigation scheme, press x to kill a session, or pane or window then y to confirm. You can also ...

https://youtu.be/XucVVgGmesM I see you there, trying to script out your tmux layouts. Tryig to get each project setup just perfect with a script, but you keep stumbling over yourself with duplicate session error messages The has-session tmux command is a handy tool to prevent this duplicate session error message when scripting your tmux layouts. command line # [1] The command is pretty straight forward, you simply ask tmux if the session name you are looking for exists. tmux has-session -t "waylonwalker_com" How I navigate tmux in 2021 [2] for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist [3] to see all of the videos in this series. References: [1]: #command-line [2]: /tmux-nav-2021/ [3]: https://www.youtube.com/playlist?list=PLTRNG6WIHETB4reAxbWza3CZeP9KL6Bkr

https://youtu.be/Y1MYmL8ZolE Tmux list keys can be a useful tool to help remind you of what kebindings you have setup. You can search for them and scroll just like in tmux copy-mode. command line # [1] You can call list-keys from the command line but the interface is not very usable by itself. It might be nice to mix with grep or a pager in some circumstances. tmux list-keys tmux command line # [2] Running list-keys from within the tmux command line puts you into a much more pleasant copy-mode. list-keys default keybinging # [3] By default tmux comes with list-keys bound to prefix+?. bind-key ? list-keys list-keys man page # [4] You can see the additional flags provided by tmux in the man page for list-keys. list-keys [-1aN] [-P prefix-string -T key-table] [key] (alias: lsk) List key bindings. There are two forms: the default lists keys as bind-key commands; -N lists only keys with attached notes and shows only the ke y and note for each key. With the default form,...

https://youtu.be/mvgM8UH11_U The tmux status bar can be a handy tool to remind yourself where you are within tmux. It can also include a bunch of system information like battery status, cpu, mem, whatever you can get from the command line. Honestly I like to keep it minimal, and actually keep it turned off most of the time. I find that it helps a little bit for others to follow along if I keep it on in certain circumstances. show the status bar # [1] You can set a hotkey to show or hide the status bar. bind s set-option -g status bind C-s set-option -g status setting the background transparent # [2] I really want a minimal status bar with very little bling, I want it to get out of the way an not draw too much attention, so step one is to set the background to transparent. # default statusbar colors #―――――――――――――――――――――――――――――――― set-option -g status-bg default set-option -g status-fg colour240 setting default colors # [3] I want my status bar to somewhat match the rest of ...

https://youtu.be/dDq0depPrTs So you have been tricking out that .tmux.conf, you’re looking for a silky smooth workflow that lets you fly through tmux with super speed, but every time you tweak out that .tmux.conf you have to restart your whole session. Not amymore, Let’s add this to the bottom of our tmux.conf so that you can see everytime it gets sourced. display-message "hello beautiful" command # [1] We can run this command from your shell to re-source your changed .tmux.conf tmux source-file ~/.tmux.conf It also works from the tmux command line. source-file ~/.tmux.conf tmux hotkey # [2] It’s very common to set this up as a keybinding so that you can do it easily without needing to memorize the exact command. bind -T prefix r source-file ~/.tmux.conf bind -n M-r source-file ~/.tmux.conf from vim # [3] This is my preferred way of re-sourcing my .tmux.conf. It sits quietly in the background, and I dont need to remember to do anything. If you are a vim user you can automat...

https://youtu.be/DkJ9rb85LC0 Quickly getting between tmux splits is critical skill for productivity. You can get by with next or prev session for awhile, but if you have more than about three session you need something a bit more targeted. Full Screen selector # [1] I have used this fzf one keybinding for quite awhile, honestly I did not make it up, and cannot remember where it came from. It will open up a session picker in a new full screen window. bind C-j new-window -n "session-switcher" "\ tmux list-sessions -F '#{?session_attached,,#{session_name}}' |\ sed '/^$/d' |\ fzf --reverse --header jump-to-session --preview 'tmux capture-pane -pt {}' |\ xargs tmux switch-client -t" Popup selector # [2] Like with many of my keybindings I have swapped this one out for a popup version. It just feels so smooth. bind C-j display-popup -E "\ tmux list-sessions -F '#{?session_attached,,#{session_name}}' |\ sed '/^$/d' |\ fzf --reverse --header jump-to-session --preview 'tmux capt...

https://youtu.be/8kZnjHPYnKU Now that we are splitting up work into their own sessions, lets talk about how to navigate between them without the command line. Navigating sessions is what kept me using a too many splits and windows workflow for far too long. It was rough, I was always tripping over panes and windows until I got too frustrated and just ran tmux kill-server to start on a new blank slate. bind -n M-P switch-client -p bind -n M-N switch-client -n How I navigate tmux in 2021 [1] for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist [2] to see all of the videos in this series. References: [1]: /tmux-nav-2021/ [2]: https://www.youtube.com/playlist?list=PLTRNG6WIHETB4reAxbWza3CZeP9KL6Bkr

https://youtu.be/c-a2Bnv_NJ0 Scripting tmux to open up specific applications can be intimidating your first time. It can be tricky to get it to start in the right directory. If you are trying to assign applictaions to a keybinding it can be easy to mess up and have weird things happen every time your ~/.tmux.conf gets sourced. Open htop in an above split # [1] I used this one for a number of years to take a quick peek into my systems performance while a memory intensive task was running. bind -n M-t split-window htop \; swap-pane -U 🗒️ note that the swap-pane -U will make the htop split active immediately Open htop in a popup # [2] With the new tmux popup windows I really like the flow of just peeking at htop in a popup and jumping back into what I was doing. It can have a more consistennt look, and not mess with the window layouts. bind -n M-t popup -E -h 95% -w 95% -x 100% "htop" Open an applicaiton in the current directory # [3] One thing that can be tricky is getting ap...

https://youtu.be/WRLRiQDjVIA So you have been working on your tmux workflow, you’ve dropped a too many window workflow for scoping work that belongs together into separate sessions, but you cannot remember what session your work is in. If your diligent you have named your window when you created it, but sometimes its intent has changed or your were just plain too lazy at the time for the extra characters needed to name it. Don’t worry we can still give that session a descriptive name. Let’s rename some sessions in the terminal. # rename the current session to me tmux rename-session me # rename the me session to scratch tmux rename-session -t me scratch There is a default keybinding that you can use <prefix>+$ to rename the current session in the tmux command line. bind-key $ command-prompt -I #S "rename-session '%%'" I’ve also had this keybinding kicking around for years, but I rarely use it anymore. You will see why in an upcoming video. bind -n M-W command-prompt "rename-ses...

https://youtu.be/5KE7Il7SOEk This is something that I made up but use every single day, this is what keeps much of what is on my blog or my teams private work wiki going. I have a few very important directories that I have assigned directly to a hotkey for fast session switching. bind -n M-i new-session -A -s waylonwalker_com "cd ~/git/waylonwalker.com/ && nvim" bind i popup -E -h 95% -w 95% -x 100% "tmux new-session -A -s waylonwalker_com 'cd ~/git/waylonwalker.com/ && nvim'" bind -n M-I popup -E "tmux new-session -A -s waylonwalker_com 'cd ~/git/waylonwalker.com/ && nvim'" tmux new-session [1] This one is building off of yeserday’s new-session post, make sure you check that one out as well. How I navigate tmux in 2021 [2] for more information on how I navigate tmux, check out this full post Also check out the full YouTube tmux-playlist [3] to see all of the videos in this series. References: [1]: /tmux-new-session/ [2]: /tmux-nav-2021/ [3]: https://www.youtube.com/playlist?...