VTubers, Random Algorithms, Constraints, TUIs
Another dose of me talking about things I love!!!
Randomisation of Ports / IPs
One of the most requested features of RustScan is the ability to randomise the ports / IPs scanned.
It’s a lot easier for a firewall to detect “1, 2, 3, 4, 5…..” then it is to detect “63341, 510, 8371, 9481, 31843”.
The first thought is to use a true random number generator, but true randomness isn’t needed for port scans.
This leads us to pseudo-randomness. Now, RustScan could have used a random library (pun intended), but we didn’t. Let’s answer why.
If we needed to calculate randomised ranges of ports for every IP address, normally we’d have to create an array of all 65535 ports and randomise them.
This isn’t an issue for a single IP, but imagine we are given a range of IP addresses. Let’s say 20. We’d have to re-generate our array of ports and randomise them. That’s:
65535 * 20 = 1310700possible port randomisations. Even without every port being touched, we’d still end up with 20 randomisations we don’t need.
We’re using Linear Congruential Generator as our randomisation algorithm. It’s fast, the randomisation is so & so, but it’s fast.
The algorithm as a pretty picture
Don’t worry, I won’t explain the full algorithm here ;) But you can find it on Wikipedia here.
Because we’re using this, we can infer different orders for different IPs without storing anything besides the first pick for each IP. We don’t have to store all 65535 ports for every IP to get a random order!
To do this, we only need 1 variable. You’ll notice in the image above the algorithm has a seed. Our seed value is rather simple:
let normalized_end = end - start;
let normalized_first_pick = rng.gen_range(0, normalized_end);It’s an incredibly cool application of a random number generator. It never occurred to me that we can roll our own RNG
Shout out to Bernardo Araujo for coming up with this algorithm, I thought I’d share it as it’s a pretty cool application!
The Rise of VTubers
A couple months ago, I found myself fascinated watching Youtubers such as Omotea play Beat Saber.
Omotea is a VTuber (kind of). They do normal Youtuber things such as play video games with a cam, but their facial expressions & body movements are tracked and mapped onto, normally, an anime character.
In Omotea’s case, instead of seeing a person dancing we see an anime avatar dancing. VTubers are quite popular in the eastern world, but in the west not so much. However, they are rising meteorically to the top of the platform.
Here’s another example of an incredibly popular VTuber, Pekora.
Pekora is playing Minecraft with their “cam” in the bottom right. We can see Pekora make facial expressions or talk throughout this video, essentially living through their anime bodies, like something out of Ready Player One.
VTuber’s often have typical anime backstories. Pekora’s is that she is a rabbit from a land called PekoLand, and this backstory is heavily played throughout her streams and her videos. Some other VTubers are more relaxed and don’t have these stories.
Many VTubers are sponsored by a company, which provides the software, the music, the art, and the games to them.
These companies often have multiple VTubers working for them, and they frequently collaborate together on videos.
Anime is incredibly popular all over the world. And VTubers take is to the next level. Imagine watching an anime character, and you ask them a question through a donation and they respond to you.
VTubers are likely the closet we can ever get to real anime characters, and their intimacy clearly brings in enough money to warrant multiple companies competing in the same market.
I quite like the idea of VTubers in some aspects. They don’t have to worry so much about public fame if they never reveal their true selves.
Theory Of Constraints
Recently I had the pleasure of gaining first hand experience from the Theory of Constraints.
The theory states that:
Identify the most important & limiting factor of a project (the bottleneck) and improve upon it until it is no longer a factor.
Find bottlenecks in your project, work on them. The idea is to not work on anything that isn’t a significant bottleneck in the project.
That last sentence is very important. While working on RustScan we noticed that the actual bottleneck was different from the perceived bottleneck.
It’s easy for us to look at a large module and say:
This is the largest bottleneck in the system
But more times than not, this isn’t the case. Our basis for making this presumption is based on our own experience and what we feel rather than what we know.
This led me on a quest to find out what really were the bottlenecks of RustScan.
To do this, I used a profiling tool.
Profiling tools watch your program run, and can tell you which functions take the longest time, which use the most CPU and more. It tells you where the bottlenecks are. No more guessing!
1 │ + 63.89% 0.00% rustscan [unknown] [.] 0xffffffffffffffff
│ ▒
2 │ + 63.73% 0.00% rustscan rustscan [.] rustscan::main
│ ▒
3 │ + 63.73% 0.00% rustscan rustscan [.] futures_executor::
│ local_pool::block_on ▒
4 │ + 63.73% 0.00% rustscan rustscan [.] std::thread::local
│ ::LocalKey<T>::with ▒
5 │ + 63.73% 0.04% rustscan rustscan [.] <core::future::fro
│ m_generator::GenFuture<T> as core::future::future::Future>::poll ▒
6 │ + 63.67% 0.00% rustscan rustscan [.] <core::future::fro
│ m_generator::GenFuture<T> as core::future::future::Future>::poll ▒Here is the log from RustScan’s profile. 63% is spent in main (no biggie, it’s main after all). But other than that, we have 2 big factors in play:
block_on (async)
LocalKey<t>::with (threading
The most costly parts of RustScan is the asynchronous scanning itself.
Instead of improving some minor functions, we now know that to make RustScan faster we need to improve upon the async scanning.
If we cannot, well — perf also gives us some other hints as to what to improve:
10 │ + 20.87% 0.03% rustscan rustscan [.] socket2::socket::S
│ ocket::connect ▒
11 │ + 20.72% 0.02% rustscan libpthread-2.31.so [.] __libc_connect Mainly the socket connections, which we are improving now.
TL;DR The theory of constraints is good, but you don’t always know what the bottlenecks are by merely looking at your code. Use tools & statistics to find these for you.
Building Nice Terminal User Interfaces
Recently I had the pleasure of redesigning the terminal user interface module for RustScan. In short, a terminal user interface is the UX / design of the terminal application.
It’s easy to look at CLI only work and assume no design goes into it, but that’s simply not true.
With web design we have all sorts of tools to make information obvious. Different shapes, buttons, images and more.
But we don’t have that in the terminal. So, here’s some thoughts on designing nice terminal applications.
RustScan
Firstly, RustScan’s TUI isn’t very accessible. This is because of the ASCII banner. No worries though, there’s an option disable that (PS. if you also have an ASCII banner, please think about A11Y).
The banner displays the name of the program (in a lovely gradient), along with the slogan (Faster Nmap scanning with Rust) and then a table).
The table contains 2 links, the GitHub & the Discord.
When the user starts the program, they should understand straight away what it is.
The name (RustScan).
What it does (Faster Nmap scanning with Rust) and then a table).
The next 2 links are important. They are placed in a gradient table which contrasts the logo and the rest of the program to make them stand out. The Discord link (for support), and the GitHub page (also for support, or for finding out more about the project).
You’ll notice the quote:
“Nmap? More like slowmap 🐢”
This is because hackers love cheesy quotes.
Okay, now onto the juicy stuff. The use of icons to denote information. We have 3 icons in RustScan:
[!] - warning. Warns the user about something (and is coloured red for danger).
[~] - information. Informs the user about something (blue).
[>] - output. Outputs information to the user (green).
By colour coding the outputs, the user will be easily able to distinguish between what they likely want to ignore (information), and warnings.
If it was down to me, I’d delete the information — but it’s needed for features the user may want to use.
Ciphey
Ciphey takes the opposite approach of RustScan. Minimalism. Ciphey aims to only give the information which the user will want. No extra info like “maybe you’ll want to do this”. Only the output.
Ciphey’s output is:
A list of encryption methods the cipher used. In order, new line delimited, with extra information if needed (such as cipher keys).
The output of decryption.
Note: The only colour in Ciphey’s output is the exact output text. Whereas RustScan has multiple outputs by nature, Ciphey only has 1 — the textual output.
This is partly inspired by Google’s search engine. In Google, you enter information and you get the result you wanted back. You don’t see what happens in the background to get you that information.
The same is true for Ciphey. The only information the user needs is the output and what it was encrypted with.
Lessons
Make your output extremely obvious
If you have to output a lot of information, change the physical colour of the information to make more important information stand out.
The most important information should be printed last. This is because if you output a lot of information, you don’t want the user to scroll up to see the answer.
It’s easy to overload the user with information. Only give the user information that they want.
Zero Cost Abstractions
Rust has this really cool thing called Zero Cost Abstractions. It’s also a thing in other low level languages, but being a Python Surfer Dude I haven’t come across it before.
Zero cost abstraction is:
What you don’t use, you don’t pay for. And what you do use, you couldn’t do any better if you coded by hand.
Let’s talk about the 2 parts of this sentence.
What you don’t use, you don’t pay for.
The language shouldn’t have a global cost for a feature that isn’t used. Let’s say to use a for loop, the language needs to have some massive 1gb file that slows down everything else. If we never use a for loop, we still pay for the for loop!
And what you do use, you couldn’t do any better if you coded by hand.
Here’s the kicker.
Say you wrote some code, a function that calculated Fibonacci numbers. And you compiled this code down into assembly.
Now let’s say you hand-write assembly to do the same function — calculate Fibonacci numbers but this time in assembly.
Handwriting it in assembly would mean we would either gain no performance, or we would lose performance.
By using zero cost abstractions, we write abstracted code (not handwritten assembly) and we couldn’t do any better if we tried to hand-write assembly. Now that’s cool!
Cool mental models I like
Okay so I was looking for mental models, and I didn’t know their name so I just googled for hours and I finally came across them. Let me share with you some cool ones!
Note: I just copied these from either here or here.
The Broken Windows Theory
The Broken Windows Theory on Wikipedia
The Broken Windows Theory suggests that visible signs of crime (or lack of care of an environment) lead to further and more serious crimes (or further deterioration of the environment).
Brook’s Law
Adding human resources to a late software development project makes it later.
Metcalfe’s Law
In network theory, the value of a system grows as approximately the square of the number of users of the system.
Occam’s Razor
Entities should not be multiplied without necessity.
The simplest choice is often the best choice.
Personal News
I was initiated to become a mod of TryHackMe, a bustling hacker community with 150,000+ players (and growing every day)
Ciphey hit 2k stars <3 :)
I WAS IN DEFCON???
I published a new blog post on packing your rust packages
That’s all for this week! Cya :D
P.S. Have a great week!!!! :) <3