FatDog64-802 on nvidia 1060 M GPU running cuda toolkit 10.1

[Uten]

I needed to get my cuda development environment up and running again. After several attempts on windows 10, banging my head against some stupid .NET version issues ( I mean come on, cuda is a c/c++ environment) I gave windows the boot. Deleted 30GB og VS garbage and gave Fatdog a few hours. lo and behold, here we are. A freaking decent GPU environment on a "budget" computer.

NOTICE: Check if your GPU is a bit older it might run with one of the pre maid nvidia sfs's to save some work. Both found in the SFS manager.

This is how I got it running:

1: Get the tools.
You need the the kernel-source-x.yy.zz.sfs for your Fatdog. You also need the devx. You can get both from the Control Panel - system - SFS manager. The x.yy.zz number should match the result you get from

Code: Select all

uname -r

The cuda-toolkit contains the GPU drivers you need. So get the toolkit that coresponds to your GPU. In my case it was: cuda_10.1.168_418.67_linux.run If you only want the GUP drivers the NVIDIA-Linux-x86_64-430.40.run is a way smaller download.

2:blacklist the nouveau driver.
I got this in my grub.cfg (lickgrub.cfg).

Code: Select all

menuentry "Fatdog64 802 + devx (no save file + nox) " {
	linux /FD64-802/vmlinuz pfix=ram blacklist=nouveau savefile=none psubdir=FD64-802 extrasfs=ram:device:nvme0n1p1:/FD64-802/NVIDIA-430.40-x86_64.sfs
	initrd /FD64-802/initrd
}

The important part is: blacklist=nouveau as nouveau will block the NVIDA driver installation. My computer starts in blind mode (acka no framebuffer ) but it boots into X with the uvcvideo driver.


3: Load the devx and kernel-source sfs
Use the sfs manager or use the comandline utility

Code: Select all

load_sfs.sh --load <path/filename.sfs>

4: Start a sandbox
Start a sandbox, mount the drive containing the NVIDIA*.run file. In my case it was something like:

Code: Select all

sandbox.sh
mount /dev/nvme0n1p1 /mnt/nvme0n1p1
cd /mnt/nvme0n1p1
sh cuda_10.1.168_418.67_linux.run --help

5: Install the GPU drivers

Code: Select all

sh cuda_10.1.168_418.67_linux.run --drivers --silent

The --silent part is kindoff a workaround as the installer does not like FD's libpanel.so

6: make the driver sfs
When done you save everything and create a sfs file (select a name that fits your installation).
Make sure to move the resulting sfs to the same folder your fatdog is loaded from at boot time.

Code: Select all

sb2dir NVIDIA-Linux-x86_64-430.40
cd /tmp/NVIDIA-Linux-x86_64-430.40
mksquash $(pwd) $(pwd).sfs -noappend -comp xz -Xbcj x86
mv ../NVIDIA-Linux-x86_64-430.40.sfs /mnt/nvme0n1p1/fd64-802

Now add an entry to your bootloader. Mine is lickgrub.cfg as I used lick to get grub installed on this. But as a warning: lick failed and I had to add every entries in the lick.cfg by hand to boot the machine again

Code: Select all

menuentry "Fatdog64 802 + nvidia driver " {
	linux /FD64-802/vmlinuz blacklist=nouveau pfix=ram savefile=none psubdir=FD64-802 extrasfs=ram:device:nvme0n1p1:/FD64-802/NVIDIA-430.40-x86_64.sfs,ram:device:nvme0n1p1:/FD64-802/fd64-devx_802.sfs
	initrd /FD64-802/initrd
}

Notice that the ram: instruction in the extrasfs loads the entier sfs to ram. My computer has 16GB memory which usually is way more than I usually need. Study your options in the Fatdog help file - section: Boot options.

If all went well you should be able to reboot and run with the nvidia native drivers. This can be checked with the nvidia-smi and nvidia-settings utilities. Also do a:

Code: Select all

# lsmod|grep nvidia
nvidia_drm             40960  2
nvidia_modeset       1081344  4 nvidia_drm
nvidia              17805312  164 nvidia_modeset
drm_kms_helper        131072  1 nvidia_drm
drm                   327680  5 drm_kms_helper,nvidia_drm
# 

Code: Select all

# nvidia-smi 
Tue Aug 27 22:24:58 2019       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1060    Off  | 00000000:01:00.0  On |                  N/A |
| N/A   40C    P8     7W /  N/A |     59MiB /  6077MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0       719      G   X                                             56MiB |
|    0      7026      G   nvidia-settings                                0MiB |
+-----------------------------------------------------------------------------+
# 

If the installation failed nvidia-smi will return a no hardware found or something similar.

7: Time for the cuda environment
Make sure the kernel-source (not sure we need it, but ) and devx sfs are loaded.
Open a terminal window and start a sandbox
Mount the location of the cuda*.run file
And issue the command:

Code: Select all

sh cuda_10.1.168_418.67_linux.run --help
# echo 'Check your options'
sh cuda_10.1.168_418.67_linux.run --toolkit --samples --silent

8: A few tweaks
To get the environment correctly when we use the shortcuts to load terminals in Fatdog we need to add cuda locations to PATH and LD_LIBRARY_PATH: Check out the cuda documentation. ( /usr/local/cuda-10.1/doc/html/cuda-installation-guide-linux/index.html )
Section 7.1 mandatory actions:

Code: Select all

echo 'export PATH=/usr/local/cuda-10.1/bin:/usr/local/cuda-10.1/NsightCompute-2019.3${PATH:+:${PATH}}' >> /root/.shinit
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> /root/.shinit

NOTICE: The way Fatdog use /root/.shinit is bound to make trouble at some point because the sfs's loaded does not merge the files. They are stacked. So the last one loaded is the one that counts. We could use the <sfsroot>/tmp/sfs/autorun.sh and stream edit the changes to .shinit but that is for another post.

9: Make the sfs
Save the sandbox changes and make the sfs

Code: Select all

sb2dir.sh cuda-toolkit-10.1
cd /tmp/cuda-toolkit-10.1
mksquashfs $(pwd) $(pwd).sfs -noappend -com xz -Xbcj x86
mv ../cuda-toolkit-10.1.sfs <target folder>

Exit the sandbox ( exit or ctrl+d).

10: Testing time
Load the cuda-toolkit sfs

Code: Select all

nvcc --version
cd /root/NVIDIA_CUDA-10.1_Samples
time make -j 4 -l 5

Tweak the -j and -l numbers to fit your system.

Hopefully that's it. With minor changes this should work for other puppies to.
If no one else needs it I'm sure I will further down the road.

Happy coding
Uten ..)