Nvidia Compute + VFIO Setup

2021-01-15T00:00:00+00:00

I recently got my hands on a Nvidia RTX 2070 Super. Despite being much more powerful than my RX580, at first I was going to use it for VM passthrough only, and not for daily use. However AMD recently discontinued support for ROCm on Polaris chips so I guess I now need to use the 2070 for compute purposes such as machine learning ^{1</a></sup>.
Obviously these two scenarios require different drivers: nvidia</code> for the host
compute, and vfio-pci</code> for passthrough. Switching between the two is easy, but
requires a sequence of unbinds and binds for each of the (4!!) PCI devices that
are exposed.</p>}

To help ease this driver switching I wrote a wrote script calledpci-bind</a>, which handles the unbinding and binding a new driver for a group of devices with a certain PCI address.</p>

To bind nvidia</code> for compute I can just run:</p>

sudo pci-bind nvidia 0000:0a:00
</code></pre>
Replacing nvidia</code> with vfio-pci</code> does the opposite, which is what I put at
the top of my qemu</code> script.</p>
On my system lspci</code> gives:</p>
...
0a:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] (rev a1)
0a:00.1 Audio device: NVIDIA Corporation TU104 HD Audio Controller (rev a1)
0a:00.2 USB controller: NVIDIA Corporation TU104 USB 3.1 Host Controller (rev a1)
0a:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU104 USB Type-C UCSI Controller (rev a1)
...
</code></pre>
and each of these live in /sys/bus/pci/devices/0000:0a:00.{0-3}</code>. Hopefully you
can see where the 0000:0a:00</code> argument comes from (it's just a prefix).</p>
One issue I've experienced with this setup is that XWayland will pick up the
Nvidia drivers and hold /dev/nvidia0</code> open so that the nvidia</code> module will
refuse the unbind request. I found that this was due to glvnd</code> loading the
Nvidia EGL ICD from /usr/share/glvnd/egl_vendor.d/10_nvidia.json</code>.
Unfortunately I couldn't find a way to simply not install the Nvidia GL driver
but include the nvidia</code> kernel driver (i.e. a headless driver of some sort) on
Arch. Anyways to stop glvnd</code> from loading the nvidia</code> driver, I simply added
the following to force EGL to load mesa</code> only:</p>
export __EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/50_mesa.json
</code></pre>
I also use flatpak, which will end up trying to install the Nvidia GL drivers
as a flatpak package. To disable this I just masked out the driver package:</p>
flatpak mask "org.freedesktop.Platform.GL.nvidia*"
</code></pre>
^{1</sup>
screw Nvidia for making CUDA proprietary and not advancing open standards
like OpenCL, but also screw AMD for discontinuing ROCm support randomly on a
fairly recent card. Hopefully Intel comes up with something better, oh wait
they did and SYCL is just another</em> standard.</p>
</div>}


Site Redesign
2021-01-06T00:00:00+00:00
Wow... it's been a long time since I considered updating my website.</p>
Ever since I started using sway, I started following the sourcehut guys'
(Drew Devault</a> and Simon Ser</a>)
blogs. They have really influenced my philosophy, so I decided to redesign my
website using a static site generator (Zola</a>). The keener
eyed will notice a few stylistic similarities in the site (i.e. the layout of
the home page). No more node backend, no more authentication, no more dependencies
going out of date -- just plain HTML generated from Markdown.</p>
In addition I've tried to minimize the number of styles I'm using while keeping
things aesthetically pleasing to make things as fff</em>ast as possible. I quite
like how things turned out!</p>
I hope to continue writing, maybe once or twice a month is a reasonable target
for me.</p>
By the way, the source is available here: https://github.com/lolzballs/website</a>.</p>


Visualising Electric Fields with WebGL
2017-12-14T00:00:00+00:00
</p>
I recently saw a few blog posts (such as this</a>), which inspired me to do something with WebGL. In Physics class we just finished learning about electric and magnetic fields, and I wanted to see what electric field lines would actually look like, as opposed to something that I just drew on paper. My visualisation can be seen here</a>.</p>
Electric Field Lines</h1>
An electric field is a vector field</a>, where each point in space has a direction and magnitude attached to it. In an electric field, these vectors represent the force that a charged object (1C) would experience when placed in that position. To illustrate this vector field, electric field lines are drawn emanating from a charge, which represents the path a charge would take, given an initial position, where the line starts.</p>
My visualisation of this uses a normal 2D canvas for this (no WebGL yet!</em>). A function called getStrength</code> calculates the electric force at a given position. This calculation relies the superposition principle, where we sum up all the forces created by each charge in the simulation.</p>
Point Charges</h3>
Each point charge creates a force F: \[ F = \frac{kqq_0}{r^2} \], where \(q\) is the charge of the source, \(q_0 = 1C\), and \(r\) is the distance from the charge to the position considered. Then initial starting points for paths are created, radially dividing point charges into divisions. These paths are then calculated using Euler's method.</p>
Plate Charges</h3>
A plate charge is more difficult to calculate, as it is composed of infinitely many point charges of charge 0, which add up to a charge \(q\).
To compute the force exerted by a plate, we split up the plate into \(n\) point charges each with charge \(q \over n\), and compute the force exerted by these charges as a whole.</p>
\[ F = \sum^n_{i = 0} \frac{k\frac{q}{n}}{r_i^2} \]</p>
To extend this to 2D (forces are vectors, not scalars), we consider the vertical and horizontal components of this force separately. On each of those, we take the limit as \(n\) approaches infinity, resulting in an integral.</p>
The force parallel to the plate results in
\[ F= \frac{kq}{l} \int_0^l \frac{r - p}{((r-p^2) + d^2)^\frac{3}{2}} dp \], where \(l\) is the length of the plate, \(r\) is the distance to the left endpoint of the plate (in direction parallel to the plate), and \(d\) is the distance to the plate in the direction perpendicular to the plate. (If only I knew linear algebra and vector calculus better...)</p>
The expansion and vertical component of the force is left as an exercise to the reader.</em></p>
Euler's method</h1>
Given an initial position \(s_0\), we calculate the force \(F_0\) at that position using getStrength</code>.
By Newton's 2nd law we have \[F_0 = \frac{d^2s}{dt^2}\] for a unit mass.
After a small step \(dt\), the force will have moved this unit mass to a position \[s_1 = F_0 * dt\].
From here we calculate \(F_1\), again with getStrength</code>, where we calculate a new position \(s_2\).
This process is repeated until a limit (currently 1000 in my program), where each position \(s\) is a point on the path.</p>
Electric Potential</h1>
Finally we get to the WebGL part! Since electric potential is a scalar value, I thought it would be a cool idea to show the electric potential around these charges using colours. To do this, we use a fragment shader where each fragment is coloured by the electric potential at that point. Electric potential is similar to electric field strength, but it represents the potential energy for a unit charge at a position and thus is directly inversely proportional to the distance.</p>
\[ V = \frac{kqq_0}{r} \]</p>
The calculations are all very similar to before, but this time there are no vector components!</p>
For a plate charge the electric potential is calculated similarly to the electric field. There is only one scalar component now:</p>
\[ V = \frac{kq}{l} \int_0^l \frac{1}{\sqrt{d^2 + (r- p)^2}} dp \]</p>
From here, we can evaluate and simplify the definite integral, giving us:
\[ V = \ln\left(\frac{\sqrt{(l-r)^2 + d^2} + r - l}{\sqrt{r^2 + d^2} + r}\right) \].</p>
Using this formula for a plate charge and the previous formula for a point charge, we calculate the total electric potential \(\sum V\) at a point. This point is then assigned a colour based on sign and magnitude, and rendered to the screen, all in a fragment shader!</p>
Credits to Jacky Liao</a> who helped with a bunch of math and stuff!</p>
This is my first blog post.</p>