Just a stranger trying things.
I’m wondering, the integrated RAM like Intel did for Lunar Lake, could the same performance be achieved with the latest CAMM modules? The only real way to go integrated to get the most out of it is doing it with HBM, anything else seems like a bad trade-off.
So either you go HBM with real bandwidth and latency gains or CAMM with decent performance and upgradeable RAM sticks. But the on-chip ram like Intel did is neither providing the HBM performance nor the CAMM modularity.
They used PimEyes, nothing new.
Of importance: they do not want to release the tool but use it as a way to raise awareness.
You mean between the French article and the English comment? :)
From what I understand, bsky’s architecture seems to allow federation at multiple levels. On one side the individual profiles are actually websites and the app aggregates the content almost as an RSS reader. I do see some profiles which are independent like Jeff Gierling’s, so yes federation at the profile level seems to work.
And this is really important because it is one way to prevent your data from being hostage by the service. Then there is another level of federation. I’m not entirely sure of the terminology here, but there is one aggregator aspect, which is quite compute intensive. And that one I don’t know if there is another instance of it. But functionally speaking, I’m quite impressed by the technical aspect of bsky. There has been a lot of thought put into it.
And monetizing it is not the issue, the problem is mostly how. That they have some paid features is fine, it’s even important that there are ways to monetize it without milking their users of their privacy.
Let’s hope this works out and becomes sustainable while respecting the users!
I’m very grateful for your extended help. I’ve made some progress. I’m able to get the prompt to appear asking me for my passphrase to unlock the right partition (sda3 in my case). Entering the passphrase, however, drops me in the Dracut emergency shell after ~3min of dracut logs, seemingly looping. (Edit: the reason for why it drops me in the shell is very unclear. It says Dropping to debug shell. /bin/sh: can't access tty: job control turned off.
And if I try to exit the dracut shell, it says dracut Warning: could not boot.
).
In the Dracut emergency shell, checking /dev/mapper/
I see a luks-<sda3-uuid>
listed. Running blkid
I see it listed too with TYPE=crypto_LUKS
. I also see a dev/dm-0
with a dedicated UUID, in ext4. I ran blkid
which shows:
/dev/mapper/luks-705fc477-573a-4ef6-81b6-a14c43cda1f5: UUID="57955343-922a-4918-9bc1-797ca8d13a9c" TYPE="ext4"
/dev/sda1: UUID="cc5e0b03-3544-4bef-ab8b-8b72dd236926" TYPE="ext4"
/dev/sda2: UUID="4df1af6c-3199-4bb2-bb12-bcf897cfc6fc" TYPE="swap"
/dev/sda3: UUID="705fc477-573a-4ef6-81b6-a14c43cda1f5" TYPE="crypto_LUKS"
/dev/dm-0: UUID="57955343-922a-4918-9bc1-797ca8d13a9c" TYPE="ext4"
I checked the status of the filesystem running cryptsetup status /dev/mapper/luks-<sda3-uuid>
and it says it is active
, which I guess means it is unlocked?
I checked the /root
directory, and it is empty. So I tried to mount the partition myself: mount /dev/mapper/luks-<sda3-uuid> /root
but it fails saying mount: mounting /dev/mapper/luks-<sda3-uuid> on /root failed: No such file or directory
and that got me really puzzled? I’ve been searching far and wide but I can’t seem to find anyone with a similar situation. I feel like I’m close to getting this working.
Below is my syslinux kernel config, and the 2nd and 3rd items are what I booted into (/boot/extlinux.conf
)
# Generated by update-extlinux 6.04_pre1-r15
DEFAULT menu.c32
PROMPT 0
MENU TITLE Alpine/Linux Boot Menu
MENU HIDDEN
MENU AUTOBOOT Alpine will be booted automatically in # seconds.
TIMEOUT 10
LABEL lts
MENU DEFAULT
MENU LABEL Linux lts
LINUX vmlinuz-lts
INITRD initramfs-lts
APPEND root=/dev/mapper/root modules=sd-mod,usb-storage,ext4 cryptroot=UUID=705fc477-573a-4ef6-81b6-a14c43cda1f5 cryptdm=root rootfstype=ext4 rd.debug log_buf_len=1M rd.shell
LABEL lts
MENU DEFAULT
MENU LABEL Dracut Linux lts
LINUX vmlinuz-lts
INITRD /boot/initramfs-6.6.56-0-lts.img
APPEND root=/dev/mapper/luks-705fc477-573a-4ef6-81b6-a14c43cda1f5 modules=sd-mod,usb-storage,ext4 rootfstype=ext4 rd.shell rd.debug log_buf_len=1M rd.luks.uuid=705fc477-573a-4ef6-81b6-a14c43cda1f5
LABEL lts
MENU DEFAULT
MENU LABEL Dracut Linux lts 2
LINUX vmlinuz-lts
INITRD /boot/initramfs-6.6.56-0-lts.img
APPEND modules=sd-mod,usb-storage,ext4,dm,crypt,rootfs-block rootfstype=ext4 rootflags=rw,relatime rd.shell rd.debug log_buf_len=1M root=UUID=57955343-922a-4918-9bc1-797ca8d13a9c rd.luks.uuid=705fc477-573a-4ef6-81b6-a14c43cda1f5
And here the /proc/cmdline
of the booted partition:
BOOT_IMAGE=vmlinuz-lts modules=sd-mod,usb-storage,ext4,dm,crypt,rootfs-block rootfstype=ext4 rootflags=rw,relatime rd.shell rd.debug log_buf_len=1M root=UUID=57955343-922a-4918-9bc1-797ca8d13a9c rd.luks.uuid=705fc477-573a-4ef6-81b6-a14c43cda1f5 initrd=/boot/initramfs-6.6.56-0-lts.img
Here is my setup, when I boot in my regular initramfs (the one I’m trying to replicate using dracut):
mytestalpine:~# lsblk -o NAME,FSTYPE,FSVER,LABEL,UUID,FSAVAIL,FSUSE%,MOUNTPOINTS
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
sda
├─sda1 ext4 cc5e0b03-3544-4bef-ab8b-8b72dd236926 195.5M 21% /boot
├─sda2 swap 4df1af6c-3199-4bb2-bb12-bcf897cfc6fc [SWAP]
└─sda3 crypto_LUKS 705fc477-573a-4ef6-81b6-a14c43cda1f5
└─root ext4 57955343-922a-4918-9bc1-797ca8d13a9c 2.3G 8% /
mytestalpine:~# lsblk -l -n /dev/sda3
sda3 8:3 0 2.8G 0 part
root 253:0 0 2.8G 0 crypt /
Note: No idea of the relevance, but I’m testing this setup in a VM, with a BIOS firmware.
That does not sound like a viable long term solution to me.
Wizz
Thank you for your help. I spent time digging into this rabbit hole, and while I’ve learned a lot, I am struggling to get the basics to work. Right now, I’m focusing on being able to just boot an image I created using dracut, excluding all the initial stuff I wanted, just be able to reproduce the original functionality of being able to unlock my luks partition using my keyboard.
Where I’m at:
I am building my initramfs using the following command: dracut -f -v --add crypt --add lvm --add dm
. I get the following output log:
mytestalpine:~# dracut -f -v --add crypt --add lvm --add dm dracut[I]: Executing: /usr/bin/dracut -f -v --add crypt --add lvm --add dm dracut[I]: Module ‘dash’ will not be installed, because command ‘dash’ could not be found! dracut[I]: Module ‘mksh’ will not be installed, because command ‘mksh’ could not be found! dracut[I]: Module ‘caps’ will not be installed, because command ‘capsh’ could not be found! dracut[I]: Module ‘modsign’ will not be installed, because command ‘keyctl’ could not be found! dracut[I]: Module ‘i18n’ will not be installed, because command ‘loadkeys’ could not be found! dracut[I]: Module ‘url-lib’ will not be installed, because command ‘curl’ could not be found! dracut[I]: Module ‘btrfs’ will not be installed, because command ‘btrfs’ could not be found! dracut[I]: Module ‘dmraid’ will not be installed, because command ‘dmraid’ could not be found! dracut[I]: Module ‘dmsquash-live-ntfs’ will not be installed, because command ‘ntfs-3g’ could not be found! dracut[I]: Module ‘mdraid’ will not be installed, because command ‘mdadm’ could not be found! dracut[I]: Module ‘crypt-gpg’ will not be installed, because command ‘gpg’ could not be found! dracut[I]: Module ‘cifs’ will not be installed, because command ‘mount.cifs’ could not be found! dracut[I]: Module ‘iscsi’ will not be installed, because command ‘iscsi-iname’ could not be found! dracut[I]: Module ‘iscsi’ will not be installed, because command ‘iscsiadm’ could not be found! dracut[I]: Module ‘iscsi’ will not be installed, because command ‘iscsid’ could not be found! dracut[I]: 95nfs: Could not find any command of ‘rpcbind portmap’! dracut[I]: Module ‘nvmf’ will not be installed, because command ‘nvme’ could not be found! dracut[I]: Module ‘nvmf’ will not be installed, because command ‘jq’ could not be found! dracut[I]: Module ‘biosdevname’ will not be installed, because command ‘biosdevname’ could not be found! dracut[I]: Module ‘masterkey’ will not be installed, because command ‘keyctl’ could not be found! dracut[I]: Module ‘dash’ will not be installed, because command ‘dash’ could not be found! dracut[I]: Module ‘mksh’ will not be installed, because command ‘mksh’ could not be found! dracut[I]: Module ‘caps’ will not be installed, because command ‘capsh’ could not be found! dracut[I]: Module ‘modsign’ will not be installed, because command ‘keyctl’ could not be found! dracut[I]: Module ‘url-lib’ will not be installed, because command ‘curl’ could not be found! dracut[I]: Module ‘btrfs’ will not be installed, because command ‘btrfs’ could not be found! dracut[I]: Module ‘dmraid’ will not be installed, because command ‘dmraid’ could not be found! dracut[I]: Module ‘dmsquash-live-ntfs’ will not be installed, because command ‘ntfs-3g’ could not be found! dracut[I]: Module ‘mdraid’ will not be installed, because command ‘mdadm’ could not be found! dracut[I]: Module ‘crypt-gpg’ will not be installed, because command ‘gpg’ could not be found! dracut[I]: Module ‘cifs’ will not be installed, because command ‘mount.cifs’ could not be found! dracut[I]: Module ‘iscsi’ will not be installed, because command ‘iscsi-iname’ could not be found! dracut[I]: Module ‘iscsi’ will not be installed, because command ‘iscsiadm’ could not be found! dracut[I]: Module ‘iscsi’ will not be installed, because command ‘iscsid’ could not be found! dracut[I]: 95nfs: Could not find any command of ‘rpcbind portmap’! dracut[I]: Module ‘nvmf’ will not be installed, because command ‘nvme’ could not be found! dracut[I]: Module ‘nvmf’ will not be installed, because command ‘jq’ could not be found! dracut[I]: Module ‘masterkey’ will not be installed, because command ‘keyctl’ could not be found! dracut[I]: *** Including module: sh *** dracut[I]: *** Including module: busybox *** dracut[I]: *** Including module: crypt *** dracut[I]: *** Including module: dm *** dracut[D]: Skipping udev rule: 10-dm.rules dracut[D]: Skipping udev rule: 13-dm-disk.rules dracut[D]: Skipping udev rule: 95-dm-notify.rules dracut[D]: Skipping udev rule: 64-device-mapper.rules dracut[D]: Skipping udev rule: 60-persistent-storage-dm.rules dracut[D]: Skipping udev rule: 55-dm.rules dracut[I]: *** Including module: kernel-modules *** dracut[I]: *** Including module: kernel-modules-extra *** dracut[D]: kernel-modules-extra: configuration source “/run/depmod.d” does not exist dracut[D]: kernel-modules-extra: configuration source “/etc/depmod.d” does not exist dracut[D]: kernel-modules-extra: configuration source “/lib/depmod.d” does not exist dracut[I]: *** Including module: lvm *** dracut[D]: Skipping udev rule: 11-dm-lvm.rules dracut[D]: Skipping udev rule: 64-device-mapper.rules dracut[D]: Skipping udev rule: 56-lvm.rules dracut[D]: Skipping udev rule: 60-persistent-storage-lvm.rules dracut[I]: *** Including module: rootfs-block *** dracut[I]: *** Including module: terminfo *** dracut[I]: *** Including module: udev-rules *** dracut[D]: Skipping udev rule: 70-persistent-net.rules dracut[I]: *** Including module: usrmount *** dracut[I]: *** Including module: base *** dracut[I]: *** Including module: fs-lib *** dracut[I]: *** Including module: shutdown *** dracut[I]: *** Including modules done *** dracut[I]: *** Installing kernel module dependencies *** dracut[I]: *** Installing kernel module dependencies done *** dracut[I]: *** Resolving executable dependencies *** dracut[I]: *** Resolving executable dependencies done *** dracut[I]: *** Hardlinking files *** dracut[D]: Mode: real dracut[D]: Method: sha256 dracut[D]: Files: 457 dracut[D]: Linked: 0 files dracut[D]: Compared: 0 xattrs dracut[D]: Compared: 6 files dracut[D]: Saved: 0 B dracut[D]: Duration: 0.015759 seconds dracut[I]: *** Hardlinking files done *** dracut[I]: Could not find ‘strip’. Not stripping the initramfs. dracut[I]: *** Generating early-microcode cpio image *** dracut[I]: *** Store current command line parameters *** dracut[I]: Stored kernel commandline: dracut[I]: rootfstype=ext4 rootflags=rw,relatime dracut[E]: ldconfig exited ungracefully dracut[I]: *** Creating image file ‘/boot/initramfs-6.6.56-0-lts.img’ *** dracut[I]: Using auto-determined compression method ‘gzip’ dracut[D]: Image: /var/tmp/dracut.Ds3W3x/initramfs.img: 12M dracut[D]: ======================================================================== dracut[D]: Version: dracut-060 dracut[D]: lib/dracut/dracut-060 dracut[D]: dracut[D]: Arguments: -f -v --add ‘crypt’ --add ‘lvm’ --add ‘dm’ dracut[D]: lib/dracut/build-parameter.txt dracut[D]: dracut[D]: dracut modules: dracut[D]: sh dracut[D]: busybox dracut[D]: crypt dracut[D]: dm dracut[D]: kernel-modules dracut[D]: kernel-modules-extra dracut[D]: lvm dracut[D]: rootfs-block dracut[D]: terminfo dracut[D]: udev-rules dracut[D]: usrmount dracut[D]: base dracut[D]: fs-lib dracut[D]: shutdown dracut[D]: lib/dracut/modules.txt dracut[D]: ========================================================================
<Truncanted due to char limit>
Then I updated the /boot/extlinux.conf
file, adding the following second entry (displaying the first one just for comparison):
LABEL lts
MENU DEFAULT
MENU LABEL Linux lts
LINUX vmlinuz-lts
INITRD initramfs-lts
APPEND root=/dev/mapper/root modules=sd-mod,usb-storage,ext4 cryptroot=<my-uuid> cryptdm=root quiet rootfstype=ext4
LABEL lts
MENU LABEL dracut-img
LINUX vmlinuz-lts
INITRD /boot/initramfs-6.6.56-0-lts.img
APPEND root=/dev/mapper/root modules=sd-mod,usb-storage,ext4 cryptroot=UUID=<my-uuid> cryptdm=root quiet rootfstype=ext4 rootflags=rw,relatime
I added the rootflags=rw,relatime
because this was shown in the dracut log, so I thought perhaps that mattered. But for the most part I left it the same as the previous entry, because I’m trying to do the same thing I suppose. Perhaps I’m mistaken?
The current result of booting that image leads to a long loading (not asking for the passphrase to unlock the partition) then displaying the following error:
dracut Warning: Could not boot.
dracut Warning: "/dev/mapper/root" does not exist
Generating "/run/initramfs/rdsosreport.txt"
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report.
To get more debug information in the report, reboot with "rd.debug" added to the kernel command line.
Dropping to debug shell.
Before dropping me in a shell, in which I have not found anything useful to do. I am surely missing something basic as my understanding of what’s happening is pretty superfluous.
What I’m noticing which may be of importance:
dracut[E]: ldconfig exited ungracefully
, in the dracut output log. Perhaps this matters and should be fixed? An image is nonetheless generated.device-mapper
and lvm
missing, why did dracut complain about them missing for me to compile my own image? and would I need to add options in the /boot/extlinux.conf
file, when they are not required for the original boot entry, when all I’m trying to do (as a start) is just make sure I can reproduce a bootable kernel image?whats great is that with ollama and webui, you can as easily run it all on one computer locally using the open-webui pip package or in a remote server using the container version of open-webui.
Ive run both and the webui is really well done. It offers a number of advanced options, like the system prompt but also memory features, documents for RAG and even a built in python ide for when you want to execute python functions. You can even enable web browsing for your model.
I’m personally very pleased with open-webui and ollama and they both work wonders together. Hoghly recommend it! And the latest llama3.1 (in 8 and 70B variants) and llama3.2 (in 1 and 3B variants) work very well, even on CPU only, for the latter! Give it a shot, it is so easy to set up :)
Thank you for your help.
I decided to give dracut a shot, see how far I could get.
I created a directory /usr/lib/dracut/modules.d/99usb-mount
in which I created two scripts:
A first module /usr/lib/dracut/modules.d/99usb-mount/module-setup.sh
, executable:
#!/bin/bash
check() {
return 0
}
depends() {
echo "crypt"
return 0
}
install() {
inst_hook pre-mount 90 "$moddir/usb-mount.sh"
}
And a second script /usr/lib/dracut/modules.d/99usb-mount/usb-mount.sh
, also executable:
#!/bin/bash
LUKS_PARTITION=/dev/sda3
USB_NKL=/dev/disk/by-uuid/<MY-UUID>
USB_MOUNT_DIR=/mnt/my-usb/
KEY_FILENAME=mykey.key
# Wait for the USB to be detected and available
for i in {1..10}; do
if [ -b ${USB_NKL} ]; then
break
fi
sleep 1
done
# Mount the USB stick
mount ${USB_NKL} ${USB_MOUNT_DIR}
# Check if the mount was successful
if [ $? -ne 0 ]; then
echo "Failed to mount USB stick"
exit 1
fi
# Unlock the LUKS partition using the keyfile
if [ -e "${USB_MOUNT_DIR}/${KEY_FILENAME}" ]; then
cryptsetup luksOpen "${LUKS_PARTITION}" cryptroot --key-file "${USB_MOUNT_DIR}/${KEY_FILENAME}"
else
echo "Keyfile not found!"
echo "Failed to unlock LUKS partition"
exit 1
fi
I then fixed some dependencies and got around installing device-mapper
, providing dmsetup
, required by dm
, required by crypt
, required by my scripts.
Then I ran: dracut -f
, which didn’t seem to have any issue and includes my module:
[...]
dracut[I]: *** Including module: usb-mount ***
[...]
dracut[E]: ldconfig exited ungracefully
[...]
dracut[I]: *** Creating initramfs image file '/boot/initramfs-6.6.54-0-lts.img' done ***
Not sure if this ldconfig
error should be of any concern? The end image seems to have been created successfully.
When I check the verbose output, I see my module being included:
dracut[D]: -rwxr-xr-x 0/0 747 2024-10-07 22:30:00 lib/dracut/hooks/pre-mount/90-usb-mount.sh
However, it is here numbered 90 when above I had placed it in 99, no idea what that’s about? (edit: actually I wrote 90 in the module-setup.sh
, so this is normal I guess).
Then I rebooted with my key and the prompt for my password to unlock my LUKS partition still appeared.
In the kernel messages I see my usb stick being detected (perhaps not mounted?) prior to the password prompt, so not sure what’s going on. Do you see any issue with my attempt? Or would you happen to have any propositions for debugging this further? I’m a bit lost as to how I can diagnose the issue.
Here are the kernel messages regarding the usb detection and a few seconds later, me unlocking the LUKS partition:
[ 1.748076] usb 1-1: new high-speed USB device number 2 using xhci_hcd # usb 1-1 / sdb is my USBkey. It seems to have been detected but not mounted?
[ 2.068060] device-mapper: uevent: version 1.0.3
[ 2.068190] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@redhat.com
[ 2.078157] Key type encrypted registered
[ 2.153792] usb 1-1: New USB device found, idVendor=067b, idProduct=2517, bcdDevice= 1.00
[ 2.153799] usb 1-1: New USB device strings: Mfr=1, Product=4, SerialNumber=6
[ 2.153801] usb 1-1: Product: ClipDrive
[ 2.153803] usb 1-1: Manufacturer: BUFFALO
[ 2.153805] usb 1-1: SerialNumber: A9200502030000221
[ 2.155494] usb-storage 1-1:1.0: USB Mass Storage device detected
[ 2.157341] scsi host3: usb-storage 1-1:1.0
[ 2.159772] usbcore: registered new interface driver uas
[ 3.221531] scsi 3:0:0:0: Direct-Access BUFFALO ClipDrive 1.00 PQ: 0 ANSI: 0 CCS
[ 3.224250] sd 3:0:0:0: [sdb] 507904 512-byte logical blocks: (260 MB/248 MiB)
[ 3.227885] sd 3:0:0:0: [sdb] Write Protect is off
[ 3.227899] sd 3:0:0:0: [sdb] Mode Sense: 23 00 00 00
[ 3.231635] sd 3:0:0:0: [sdb] No Caching mode page found
[ 3.231645] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[ 3.247551] sd 3:0:0:0: [sdb] Attached SCSI removable disk
[ 6.323670] EXT4-fs (dm-0): orphan cleanup on readonly fs # the 3 seconds gap is me unlocking the LUKS using the keyboard
[ 6.323954] EXT4-fs (dm-0): mounted filesystem 33a8b408-37ff-4b8a-98bf-bba8b6f00604 ro with ordered data mode. Quota mode: none.
[ 6.324134] Mounting root: ok.
It seems you might be right. There is so little documentation for initramfs in Alpine Linux (the wiki page is very barebones), but I did manage to find this open issue:
https://gitlab.alpinelinux.org/alpine/mkinitfs/-/issues/18
So I guess this confirms that it is not yet possible.
Could you expand on your suggestion with customizing the init script? Where is this file located, and would you have some pointers of how to get started to customize it for my use case?
I wouldn’t assume this is done with malice in mind, but maybe this is someone unaware of the importance of a formal license.