From Newsgroup: comp.sys.raspberry-pi

My "PC" for most daily use, is a raspi 5 with updated Bookworm, running
on a SSD connected to USB via a SATA-adapter. I am tempted to change to
NVME to get the last bit of latency away.
But what is the easiest way to copy the system to NVME? I see 3
possibilities:

1: Connect SSD and NVME to a Windows PC and use Macrium Reflect to
move the system.
2: Connect NVME to raspi, boot the system from SSD, and somehow
copy the running system to NVME.
3: Connect NVME to raspi, boot from SD-card and copy the system from
SSD to NVME. But how?

By now option 1, is the only one I can manage and the one used to move
the system from SD-card to SSD. Searching for help relating to options 2
and 3 on the net ends in quite complicated solutions that maybe will not
work in bookworm. So if you see an easy solution that a Windows idiot
can manage, please come forward :-)

Below is information showing what the system on the SSD looks like. The listing is taken when booted to a SD-card with a fresh Bookworm install:

raspberrypi@raspberrypi:~ $ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.8G 0 3.8G 0% /dev
tmpfs 805M 6.2M 799M 1% /run
/dev/mmcblk0p2 57G 5.0G 49G 10% /
tmpfs 4.0G 368K 4.0G 1% /dev/shm
tmpfs 5.0M 48K 5.0M 1% /run/lock
/dev/mmcblk0p1 510M 75M 436M 15% /boot/firmware
tmpfs 805M 160K 805M 1% /run/user/1000
/dev/sda2 234G 19G 203G 9% /media/raspberrypi/rootfs
/dev/sda1 511M 76M 436M 15% /media/raspberrypi/bootfs


raspberrypi@raspberrypi:~ $ sudo cat /proc/mounts
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,relatime 0 0
udev /dev devtmpfs
rw,nosuid,relatime,size=3945808k,nr_inodes=246613,mode=755 0 0
devpts /dev/pts devpts
rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=824160k,mode=755 0 0 /dev/mmcblk0p2 / ext4 rw,noatime 0 0
securityfs /sys/kernel/security securityfs
rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4753 0 0
tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0
mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
/dev/mmcblk0p1 /boot/firmware vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro
0 0
ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=824144k,nr_inodes=206036,mode=700,uid=1000,gid=1000
0 0
gvfsd-fuse /run/user/1000/gvfs fuse.gvfsd-fuse rw,nosuid,nodev,relatime,user_id=1000,group_id=1000 0 0
portal /run/user/1000/doc fuse.portal rw,nosuid,nodev,relatime,user_id=1000,group_id=1000 0 0
/dev/sda2 /media/raspberrypi/rootfs ext4 rw,nosuid,nodev,relatime,errors=remount-ro 0 0
/dev/sda1 /media/raspberrypi/bootfs vfat rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,showexec,utf8,flush,errors=remount-ro
0 0
raspberrypi@raspberrypi:~ $
--
Jesper
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 31/07/2024 08:51, Jesper wrote:

My "PC" for most daily use, is a raspi 5 with updated Bookworm, running
on a SSD connected to USB via a SATA-adapter. I am tempted to change to
NVME to get the last bit of latency away.
But what is the easiest way to copy the system to NVME? I see 3 possibilities:

   1: Connect SSD and NVME to a Windows PC and use Macrium Reflect to move the system.
   2: Connect NVME to raspi, boot the system from SSD, and somehow copy the running system to NVME.
   3: Connect NVME to raspi, boot from SD-card and copy the system from SSD to NVME. But how?

It's not Windows, you don't need 3rd party tools to do simple things
such as copying files to new discs, everything you need is provided,
although it may not be obvious what to do.

Option 3 is the best, as both the drives can be connected to the Pi,
booting from an OS image the SD card allows you to perform the copy.

As long as your new NVME is larger than the SSD, you can just do low
level copy with the dd command, then resize the rootfs partition on the
new drive to use any extra space with the gparted program (if not
installed use: apt install gparted).

e.g. dd if=/dev/sda of=/dev/nvme0n1 bs=1M status=progress

Use fdisk -l to find the nvme disk name as it may not be as above.

Alternatively use gparted to create a 512MB FAT boot partition, and the
rest of the drive as a ext4 root partition, then mount them, and copy
files from the boot and root partitions with rsync.

e.g.

mkdir -p /mnt/SSD/boot
mount /dev/sda1 /mnt/SSD/boot
mkdir -p /mnt/SSD/root
mount /dev/sda2 /mnt/SSD/root

mkdir -p /mnt/NVME/boot
mount /dev/nvme0n1p1 /mnt/NVME/boot
mkdir -p /mnt/NVME/root
mount /dev/nvme0n1p2 /mnt/NVME/root

rsync -vax /mnt/SSD/boot/ /mnt/NVME/boot
rsync -vaxHAX /mnt/SSD/root/ /mnt/NVME/root

Make sure you look up what those commands do and have the correct
parameters, rather than just cutting and pasting.

---druck
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Wed, 31 Jul 2024 09:51:00 +0200, Jesper wrote:

3: Connect NVME to raspi, boot from SD-card and copy the system from
SSD to NVME. But how?

Use rsync to do a file-level copy. Then the difference in volume sizes
won’t matter.

I haven’t done this on ARM, only on x86 machines (several times), but the procedure should be similar: after copying the installation across, you
will need to fix up /etc/fstab for the changed filesystem IDs, and also reinstall the bootloader. Then you should be good to go.
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 31/07/2024 23:28, Lawrence D'Oliveiro wrote:

On Wed, 31 Jul 2024 09:51:00 +0200, Jesper wrote:

3: Connect NVME to raspi, boot from SD-card and copy the system from
SSD to NVME. But how?

Use rsync to do a file-level copy. Then the difference in volume sizes won’t matter.

I haven’t done this on ARM, only on x86 machines (several times), but the procedure should be similar: after copying the installation across, you
will need to fix up /etc/fstab for the changed filesystem IDs, and also reinstall the bootloader. Then you should be good to go.

If you can create an image - there are various tools to do that, then
use the Raspberry Pi imager onto the NVME drive. That is what I did to
move from a USB SSD to NVME, when I finally got an adapter
--
Chris

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 31/07/2024 21:33, druck wrote:

As long as your new NVME is larger than the SSD, you can just do low
level copy with the dd command, then resize the rootfs partition on the
new drive to use any extra space with the gparted program (if not
installed use: apt install gparted).

That will at least get over the problem of having a different partuuid etc

Is it possible to then create a DIFFERENT partition on the empty part of
the new disk? And mount it as - say - /home?

I need to do this myself, although the urgency is very low
--
“It is dangerous to be right in matters on which the established
authorities are wrong.”

― Voltaire, The Age of Louis XIV

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 31.07.2024 22:33, druck wrote:

On 31/07/2024 08:51, Jesper wrote:

    3: Connect NVME to raspi, boot from SD-card and copy the system
from SSD to NVME. But how?

It's not Windows, you don't need 3rd party tools to do simple things
such as copying files to new discs, everything you need is provided, although it may not be obvious what to do.

Option 3 is the best, as both the drives can be connected to the Pi,
booting from an OS image the SD card allows you to perform the copy.

As long as your new NVME is larger than the SSD, you can just do low
level copy with the dd command, then resize the rootfs partition on the
new drive to use any extra space with the gparted program (if not
installed use: apt install gparted).

e.g. dd if=/dev/sda of=/dev/nvme0n1 bs=1M status=progress

Use fdisk -l to find the nvme disk name as it may not be as above.

---druck

Thank you very much to all for taking your time to reply.

By now I think the first solution from druck will be my first try (when
I get the NVME). I have before been looking at creating an image (as
suggested by Chris Townley today) , but ran in to things I did not
understand, or was ment for an earlier version of raspberry pi os. Then
i posted the question here.

So if we look at the df -h listing from my first post: raspberrypi@raspberrypi:~ $ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.8G 0 3.8G 0% /dev
tmpfs 805M 6.2M 799M 1% /run
/dev/mmcblk0p2 57G 5.0G 49G 10% /
tmpfs 4.0G 368K 4.0G 1% /dev/shm
tmpfs 5.0M 48K 5.0M 1% /run/lock
/dev/mmcblk0p1 510M 75M 436M 15% /boot/firmware
tmpfs 805M 160K 805M 1% /run/user/1000
/dev/sda2 234G 19G 203G 9% /media/raspberrypi/rootfs
/dev/sda1 511M 76M 436M 15% /media/raspberrypi/bootfs

Then the system to copy is on the 2 last lines. Correct?

And following drucks first suggestion I should run these 2 commands:
1: dd if=/dev/sda2 of=/dev/nvme0n1 bs=1M status=progress
and
2: dd if=/dev/sda1 of=/dev/nvme0n1 bs=1M status=progress

replacing the name of the NVME to what I see when it is installed on the raspi.

Best regards, and thank you for the help.
--
Jesper
--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 12:31, Jesper wrote:

Then the system to copy is on the 2 last lines. Correct?

No.

In order to preserve the partition information *you must dd the raw disk*

dd id=/dev/sda od = /dev/nvme0n1 (or whatever)

That will create a two partition disk with the UUIDS of the partitions
the same as is mentioned in the boot data: If they don't match it wont boot.

Viz:


df -h | grep ^/dev/
/dev/root 15G 1.5G 13G 11% /
/dev/mmcblk0p1 255M 51M 205M 20% /boot

These are the TWO partitions on a bootable PI device
In /boot which is DOS style formatted will be instructions on how to
boot the main system

In the main bootable system there will be the fstab file which needs to
tally with the partition ids.

more /etc/fstab:

proc /proc proc defaults 0 0 PARTUUID=b8c9fbb7-01 /boot vfat defaults 0 2 PARTUUID=b8c9fbb7-02 / ext4 defaults,noatime 0 1

In the BOOT partition is this file
more *.txt
::::::::::::::
cmdline.txt
::::::::::::::
console=serial0,115200 console=tty1 root=PARTUUID=b8c9fbb7-02
rootfstype=ext4 fs
ck.repair=yes rootwait modules-load=dwc2,g_ether

Unless the bootloader finds that partition ID, it will *not load Linux*

AIUI the boot sequence is this:

Look for a DOS style VFAT partition on SD card, then USB, then NVME.

Look for a file named 'cmdline.txt' parse the root partition ID and
attempt to load a linux image from the boot partition and have the
kernel image mount the aforementioned PARTUUID as root partition.

If the PARTUUIDs don't match, the boot sequence hangs

So it is important to have the same PARTUUID in /boot/cmdline.txt, and
in /etc/fstab, and in the partition label on the boot partition

The easy way to do this is not to clone the partitions, but the RAW DISK

And following drucks first suggestion I should run these 2 commands:
1: dd if=/dev/sda2 of=/dev/nvme0n1 bs=1M status=progress
and
2: dd if=/dev/sda1 of=/dev/nvme0n1 bs=1M status=progress

replacing the name of the NVME to what I see when it is installed on the raspi.

Best regards, and thank you for the help.

That wont work, but you wont destroy anything by trying.
--
Outside of a dog, a book is a man's best friend. Inside of a dog it's
too dark to read.

Groucho Marx



--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01.08.2024 13:55, The Natural Philosopher wrote:

On 01/08/2024 12:31, Jesper wrote:

Then the system to copy is on the 2 last lines. Correct?

No.

In order to preserve the partition information *you must dd the raw disk*

dd id=/dev/sda od = /dev/nvme0n1 (or whatever)

That will create a two partition disk with the UUIDS of the partitions
the same as is mentioned in the boot data: If they don't match it wont
boot.

Are you saying that this command (the one right above) will clone the
whole disc, and the job is done? Probably not. When I run "dd --help" on
the system i want to clone, there is no information about the switch
"id" you use in the command.

Viz:

df -h | grep ^/dev/

Running that command on my system gives:
raspberrypi@raspberrypi:~ $ df -h | grep ^/dev/
/dev/sda2 234G 19G 203G 9% /
/dev/sda1 511M 76M 436M 15% /boot/firmware
For /dev/sda1 it says "firmware", so it probably should/can not be
copied, and is permanent on the raspi5-system

/dev/root        15G  1.5G   13G  11% /
/dev/mmcblk0p1  255M   51M  205M  20% /boot

These are the TWO partitions on a bootable PI device
In /boot which is DOS style formatted will be instructions on how to
boot the main system

In the main bootable system there will be the fstab file which needs to tally with the partition ids.

more /etc/fstab:

proc            /proc           proc    defaults          0       0
PARTUUID=b8c9fbb7-01  /boot           vfat    defaults          0       2
PARTUUID=b8c9fbb7-02  /               ext4    defaults,noatime  0       1

In the BOOT partition is this file
 more *.txt
::::::::::::::
cmdline.txt
::::::::::::::
console=serial0,115200 console=tty1 root=PARTUUID=b8c9fbb7-02 rootfstype=ext4 fs
ck.repair=yes rootwait modules-load=dwc2,g_ether

Unless the bootloader finds that partition ID, it will *not load Linux*

AIUI the boot sequence is this:

Look for a DOS style VFAT partition on SD card, then USB, then NVME.

Look for a file named 'cmdline.txt' parse the root partition ID  and attempt to load a linux image from the boot  partition and have the
kernel image mount the aforementioned PARTUUID as root partition.

If the PARTUUIDs don't match, the boot sequence hangs

So it is important to have the same PARTUUID in /boot/cmdline.txt, and
in /etc/fstab, and in the partition label on the boot partition

The easy way to do this is not to clone the partitions, but the RAW DISK

And following drucks first suggestion I should run these 2 commands:
1: dd if=/dev/sda2 of=/dev/nvme0n1 bs=1M status=progress
and
2: dd if=/dev/sda1 of=/dev/nvme0n1 bs=1M status=progress

replacing the name of the NVME to what I see when it is installed on
the raspi.

Best regards, and thank you for the help.

That wont work, but you wont destroy anything by trying

So I will not try :-)

I think I should not take any more of your time for this project. Using
a PC and a windows program (Macrium Reflect) to clone the disc will take
maybe 30 minutes. Learning the ins and outs of how it should be done in raspi-os can take 14 days for a windows-addict:-)

Best regards, and thank you for the help so far.
--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Thu, 1 Aug 2024 17:47:32 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

On 01.08.2024 13:55, The Natural Philosopher wrote:

On 01/08/2024 12:31, Jesper wrote:

Then the system to copy is on the 2 last lines. Correct?

No.

In order to preserve the partition information *you must dd the raw
disk*

dd id=/dev/sda od = /dev/nvme0n1 (or whatever)

That will create a two partition disk with the UUIDS of the partitions
the same as is mentioned in the boot data: If they don't match it wont
boot.

Are you saying that this command (the one right above) will clone the
whole disc, and the job is done? Probably not. When I run "dd --help" on
the system i want to clone, there is no information about the switch
"id" you use in the command.

That command is wrong (or for an odd version of dd - but I think
it's a finger slip) make that:

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

The add bs=1m will probably work wonders for performance.
--
Steve O'Hara-Smith
Odds and Ends at http://www.sohara.org/
For forms of government let fools contest
Whate're is best administered is best - Alexander Pope
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 16:47, Jesper wrote:

On 01.08.2024 13:55, The Natural Philosopher wrote:

On 01/08/2024 12:31, Jesper wrote:

Then the system to copy is on the 2 last lines. Correct?

No.

In order to preserve the partition information *you must dd the raw disk*

dd id=/dev/sda od = /dev/nvme0n1 (or whatever)

That will create a two partition disk with the UUIDS of the partitions
the same as is mentioned in the boot data: If they don't match it wont
boot.

Are you saying that this command (the one right above) will clone the
whole disc, and the job is done? Probably not. When I run "dd --help" on
the system i want to clone, there is no information about the switch
"id" you use in the command.

Soory. old brain.
dd if=/dev/sda of = /dev/nvme0n1 (or whatever)

Viz:


df -h | grep ^/dev/

Running that command on my system gives:
raspberrypi@raspberrypi:~ $ df -h | grep ^/dev/
/dev/sda2           234G   19G   203G    9% / /dev/sda1           511M   76M   436M   15% /boot/firmware For /dev/sda1 it says "firmware", so it probably should/can not be
copied, and is permanent on the raspi5-system

It has to be copied because it contains the boot information

/dev/root        15G  1.5G   13G  11% /
/dev/mmcblk0p1  255M   51M  205M  20% /boot

These are the TWO partitions on a bootable PI device
In /boot which is DOS style formatted will be instructions on how to
boot the main system

In the main bootable system there will be the fstab file which needs
to tally with the partition ids.

more /etc/fstab:

proc            /proc           proc    defaults          0       0
PARTUUID=b8c9fbb7-01  /boot           vfat    defaults          0       2
PARTUUID=b8c9fbb7-02  /               ext4    defaults,noatime  0       1

In the BOOT partition is this file
  more *.txt
::::::::::::::
cmdline.txt
::::::::::::::
console=serial0,115200 console=tty1 root=PARTUUID=b8c9fbb7-02
rootfstype=ext4 fs
ck.repair=yes rootwait modules-load=dwc2,g_ether

Unless the bootloader finds that partition ID, it will *not load Linux*

AIUI the boot sequence is this:

Look for a DOS style VFAT partition on SD card, then USB, then NVME.

Look for a file named 'cmdline.txt' parse the root partition ID  and
attempt to load a linux image from the boot  partition and have the
kernel image mount the aforementioned PARTUUID as root partition.

If the PARTUUIDs don't match, the boot sequence hangs

So it is important to have the same PARTUUID in /boot/cmdline.txt, and
in /etc/fstab, and in the partition label on the boot partition

The easy way to do this is not to clone the partitions, but the RAW DISK

And following drucks first suggestion I should run these 2 commands:
1: dd if=/dev/sda2 of=/dev/nvme0n1 bs=1M status=progress
and
2: dd if=/dev/sda1 of=/dev/nvme0n1 bs=1M status=progress

replacing the name of the NVME to what I see when it is installed on
the raspi.

Best regards, and thank you for the help.

That wont work, but you wont destroy anything by trying

So I will not try :-)

I think I should not take any more of your time for this project. Using
a PC and a windows program (Macrium Reflect) to clone the disc will take maybe 30 minutes. Learning the ins and outs of how it should be done in raspi-os can take 14 days for a windows-addict:-)

dd will clone it in a lot less time

And that is the point, You MUST clone the WHOLE DISK. Not the partitions inside it

Best regards, and thank you for the help so far.

--
In todays liberal progressive conflict-free education system, everyone
gets full Marx.

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 17:10, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 17:47:32 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

On 01.08.2024 13:55, The Natural Philosopher wrote:

On 01/08/2024 12:31, Jesper wrote:

Then the system to copy is on the 2 last lines. Correct?

No.

In order to preserve the partition information *you must dd the raw
disk*

dd id=/dev/sda od = /dev/nvme0n1 (or whatever)

That will create a two partition disk with the UUIDS of the partitions
the same as is mentioned in the boot data: If they don't match it wont

>boot.

Are you saying that this command (the one right above) will clone the
whole disc, and the job is done? Probably not. When I run "dd --help" on
the system i want to clone, there is no information about the switch
"id" you use in the command.

That command is wrong (or for an odd version of dd - but I think
it's a finger slip) make that:

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

The add bs=1m will probably work wonders for performance.

+1

Not odd command, just brainfade. if=input file, I was thinking
InputDevice...
--
Outside of a dog, a book is a man's best friend. Inside of a dog it's
too dark to read.

Groucho Marx



--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01.08.2024 18:10, Ahem A Rivet's Shot wrote:

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

The add bs=1m will probably work wonders for performance.

On 01.08.2024 17:47, Jesper wrote:

raspberrypi@raspberrypi:~ $ df -h | grep ^/dev/
/dev/sda2 234G 19G 203G 9% /
/dev/sda1 511M 76M 436M 15% /boot/firmware
For /dev/sda1 it says "firmware", so it probably should/can not be
copied, and is permanent on the raspi5-system

The Natural philosopher says to clone both sda1 and sda2. I still wonder
if sda1 should be cloned. It is listed as "firmware". That sounds to me
like it is on a flashmemory directly on the raspi5, and you modify it
with raspi-config->Advanced options->Boot order.

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m
dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

Comments?

Best regards
--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 18:50, Jesper wrote:

On 01.08.2024 18:10, Ahem A Rivet's Shot wrote:

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

    The add bs=1m will probably work wonders for performance.

On 01.08.2024 17:47, Jesper wrote:

raspberrypi@raspberrypi:~ $ df -h | grep ^/dev/ /dev/sda2           234G   19G   203G    9% / /dev/sda1           511M   76M   436M   15% /boot/firmware For /dev/sda1 it says "firmware", so it probably should/can not be
copied, and is permanent on the raspi5-system

The Natural philosopher says to clone both sda1 and sda2. I still wonder
if sda1 should be cloned. It is listed as "firmware".

Yes. not only must it be cloned, because it contains instructions as to
what patition the linux systren is in, but the linux partiton id must be cloned as wqell and that does not exost ionside the partibin, but on the
raw disk
as well so that it has that partition number

That sounds to me

like it is on a flashmemory directly on the raspi5, and you modify it
with raspi-config->Advanced options->Boot order.

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m
dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

The second dd will wipe out the first. Dont waste my time, come back
when you have demonstrated that it doesn't' work

I've been there and have the T-shirt

That's how I know what you have to do.

/boot in the older release is /boot/firmware in the new.

My Pi4Bookworn SSD that does boot has two partitionb on it


mounted on my desktop they show this

/dev/sdb1 510M 61M 450M
12% /media/leo/bootfs
/dev/sdb2 110G 5.7G 99G
6% /media/leo/rootfs

Bootfs is what gets mounted on /boot/firmware
rootfs is a traditional Linux filesystem

In bootfs the cmdline.txt specifies

/media/leo/bootfs$ more cmdline.txt
console=serial0,115200 console=tty1 root=PARTUUID=778a9e44-02
rootfstype=ext4 fsck.repair=yes rootwait noswap=1

So that specifies the PARTUUID of the partition that *must* be mounted
as root


That PARTUUIDs are shown a s follows


eo@Juliet:/media/leo/bootfs$ sudo blkid /dev/sdb
/dev/sdb: PTUUID="778a9e44" PTTYPE="dos"
leo@Juliet:/media/leo/bootfs$ sudo blkid /dev/sdb1
/dev/sdb1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="5DF9-E225"
TYPE="vfat" PARTUUID="778a9e44-01"
leo@Juliet:/media/leo/bootfs$ sudo blkid /dev/sdb2
/dev/sdb2: LABEL="rootfs" UUID="3b614a3f-4a65-4480-876a-8a998e01ac9b" TYPE="ext4" PARTUUID="778a9e44-02"


The partition IDs are not stored on the partitions, but in the partition table. This is why you have to clone the whole disk, to clone that
partition table as well.

After booting linux will mount the original bootfs on /boot/firmware
(it used to be /boot pirior to bookworm) and the partuuid specified in cmdline.txt as the root partition

The bootfs will be re-mounted according to what is in fstab

/etc$ more fstab
proc /proc proc defaults 0 0 PARTUUID=778a9e44-01 /boot/firmware vfat defaults 0 2 PARTUUID=778a9e44-02 / ext4 defaults,noatime 0 1

All these ducks have to be lined up in a row or the bloody thing will
not boot


--
Climate Change: Socialism wearing a lab coat.

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m

This will copy sda2 to nvme0n1

dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

This will overwrite that copy with sda1. You would probably be
better off cloning sda as a whole - that will get all partitions, boot
sectors etc.
--
Steve O'Hara-Smith
Odds and Ends at http://www.sohara.org/
For forms of government let fools contest
Whate're is best administered is best - Alexander Pope
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 19:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m

This will copy sda2 to nvme0n1

dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

This will overwrite that copy with sda1. You would probably be
better off cloning sda as a whole - that will get all partitions, boot sectors etc.

+1.

That will boot, but have spare disk space.
You can extend the root partition using the Pi toolset, or I hope
(havent tried it yet) create another partition in the spare disk space....
--
The theory of Communism may be summed up in one sentence: Abolish all
private property.

Karl Marx


--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m

This will copy sda2 to nvme0n1

dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

This will overwrite that copy with sda1. You would probably be
better off cloning sda as a whole - that will get all partitions, boot sectors etc.

Yes, that's also what The Natural Philosopher says in his reply. And I
know that you both of course are right.
But I still do not know what a command that clones both sda1 and sda2 to
NVME should look like. Please?

When I run ls /media i get this:
raspberrypi@raspberrypi:~ $ ls /media
raspberrypi

Could this work to clone the whole disc:
dd if=/dev/raspberrypi of=/dev/nvme0n1 bs=1m

Best regards
--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and sda2 to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m


/dev/sda is the WHOLE disk called /dev/sda
/dev/sdaX where X is a number is that PARTITION on the disk /dev/sda
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 09:20, The Natural Philosopher wrote:

On 31/07/2024 21:33, druck wrote:

As long as your new NVME is larger than the SSD, you can just do low
level copy with the dd command, then resize the rootfs partition on
the new drive to use any extra space with the gparted program (if not
installed use: apt install gparted).

That will at least get over the problem of having a different partuuid etc

Yes the dd method will avoid any changes to partuuids, where as if you
use the rsync method you will have to change the values in /etc/fstab
and also potentially /boot/cmdline.txt (/boot/firmware/cmdline.txt on Bookworm).

Is it possible to then create a DIFFERENT partition on the empty part of the  new disk? And mount it as - say - /home?

Yes, you can do this with gparted, move the existing /home to the new partition and edit /etc/fstab to contain a line similar to:-

PARTUUID=abcd1234-03 /home ext4 nofail,noatime,errors=remount-ro 0 0

Then do a:-

mount -a

---druck

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 12:31, Jesper wrote:

By now I think the first solution from druck will be my first try (when
I get the NVME). I have before been looking at creating an image (as suggested by Chris Townley today) , but ran in to things I did not understand, or was ment for an earlier version of raspberry pi os. Then
i posted the question here.

So if we look at the df -h listing from my first post: raspberrypi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on udev            3.8G     0  3.8G   0% /dev tmpfs           805M  6.2M  799M   1% /run
/dev/mmcblk0p2   57G  5.0G   49G  10% /
tmpfs           4.0G  368K  4.0G   1% /dev/shm tmpfs           5.0M   48K  5.0M   1% /run/lock /dev/mmcblk0p1  510M   75M  436M  15% /boot/firmware tmpfs           805M  160K  805M   1% /run/user/1000 /dev/sda2       234G   19G  203G   9% /media/raspberrypi/rootfs /dev/sda1       511M   76M  436M  15% /media/raspberrypi/bootfs

Then the system to copy is on the 2 last lines. Correct?

And following drucks first suggestion I should run these 2 commands:
1: dd if=/dev/sda2 of=/dev/nvme0n1 bs=1M status=progress
and
2: dd if=/dev/sda1 of=/dev/nvme0n1 bs=1M status=progress

No that wont work, it will copy the contents of of the first partition
over the start of the nvme drive and then copy the second partition over
the top of that.

I think what you meant was to copy to /dev/nvme0n1p1 and /dev/nvme0n1p2, assuming you have already created two partitions of the correct size on
the nvme disk. (I do it like this if creating additional partitions to
enable running different OS versions, but its not needed for a straight
copy).

The single dd command I gave in my last post copies the whole of the SSD
to the nvme including the partition table and both parititions, giving
an identical drive - as long as it is bigger. The only thing to do then
is to use gparted to update the partition table and last partition to
fill the rest of that new disk.

---druck
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 18:50, Jesper wrote:

The Natural philosopher says to clone both sda1 and sda2. I still wonder
if sda1 should be cloned. It is listed as "firmware". That sounds to me
like it is on a flashmemory directly on the raspi5, and you modify it
with raspi-config->Advanced options->Boot order.

Bookworm now calls the first FAT partition on the disk which contains
device file and the kernel 'firmware' and mounts it at /boot/firmware
(previous versions just mounted at /boot). It is very much required for
the Pi to start sucessfully.

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m
dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

Comments?

Just the one:-

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

---druck
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 31/07/2024 23:28, Lawrence D'Oliveiro wrote:

On Wed, 31 Jul 2024 09:51:00 +0200, Jesper wrote:

3: Connect NVME to raspi, boot from SD-card and copy the system from
SSD to NVME. But how?

Use rsync to do a file-level copy. Then the difference in volume sizes won’t matter.

I haven’t done this on ARM, only on x86 machines (several times), but the procedure should be similar:

It does work on the Pi, I've taken and restored backup using this method
many times. As long as you use the correct flags on rsync (a bit of a
black art) all the file attributes are preserved and Linux will work.

after copying the installation across, you
will need to fix up /etc/fstab for the changed filesystem IDs, and also reinstall the bootloader. Then you should be good to go.

Yes I'd forgotten to mention that bit in my post, but don't forget to
modify cmdline.txt on the FAT partition, as that will either contain a
device name or a partition uuid of the original disc.

---druck
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and sda2
to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

I saw it, but thougt that the "sda"-part was just as an example. Did not
know at that time it is name of the whole disc. I simple have too little knowledge about handling/naming discs in linux. Last night when I wrote
the question if "raspberrypi" in the dd command would represent the
whole disc, it crossed my mind a few times that maybe sda represented
the whole disk.

But now you all have helped me arrive at the goal: One command that
fixes the whole copying. And also learned a bit.
So a big thanks to all, and sorry about my ignorance :-)

An order for the NVME, case and fan will be sent today.

Best regards

/dev/sda is the WHOLE disk called /dev/sda
/dev/sdaX where X is a number is that PARTITION on the disk /dev/sda

--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 20:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I can try running both commands:
dd if=/dev/sda2 of=/dev/nvme0n1 bs=1m

    This will copy sda2 to nvme0n1

dd if=/dev/sda1 of=/dev/nvme0n1 bs=1m

    This will overwrite that copy with sda1. You would probably be
better off cloning sda as a whole - that will get all partitions, boot
sectors etc.

Yes, that's also what The Natural Philosopher says in his reply. And I
know that you both of course are right.
But I still do not know what a command that clones both sda1 and sda2 to NVME should look like. Please?

When I run ls /media i get this:
raspberrypi@raspberrypi:~ $ ls /media
raspberrypi

Could this work to clone the whole disc:
 dd if=/dev/raspberrypi of=/dev/nvme0n1 bs=1m

 Best regards

Something like that. But..

In *nix the whole disk is normally given at connection time a device
name like /dev/sda
whereas its *partitions* would be /dev/sda1, /dev/sda2 etc etc.

when its mounted under /media it may have another name.

The mount command will show what devices you have and where they are
mounted.
So with the disks attached try

mount | grep '^/dev'

To identify the device name.

E.g. when I use the above command on as USB attached raspberry PI
bootable drive on my desktop machine I get this:

/dev/sda5 on / type ext4 (rw,relatime,errors=remount-ro)
/dev/sda1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
/dev/sdb1 on /media/leo/bootfs type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)
/dev/sdb2 on /media/leo/rootfs type ext4 (rw,nosuid,nodev,relatime,uhelper=udisks2)

This shows that the actual PC is running off /dev/sda. but the Pi disk
has been given the name /dev/sdb

The fact that its been automounted on /media is irrelevant You can only
mount *partitions* but you want the whole disk - in this case /dev/sdb2
--
The higher up the mountainside
The greener grows the grass.
The higher up the monkey climbs
The more he shows his arse.

Traditional

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01/08/2024 21:15, druck wrote:

On 01/08/2024 09:20, The Natural Philosopher wrote:

On 31/07/2024 21:33, druck wrote:

As long as your new NVME is larger than the SSD, you can just do low
level copy with the dd command, then resize the rootfs partition on
the new drive to use any extra space with the gparted program (if not
installed use: apt install gparted).

That will at least get over the problem of having a different partuuid
etc

Yes the dd method will avoid any changes to partuuids, where as if you
use the rsync method you will have to change the values in /etc/fstab
and also potentially /boot/cmdline.txt (/boot/firmware/cmdline.txt on Bookworm).

Is it possible to then create a DIFFERENT partition on the empty part
of the  new disk? And mount it as - say - /home?

Yes, you can do this with gparted, move the existing /home to the new partition and edit /etc/fstab to contain a line similar to:-

PARTUUID=abcd1234-03  /home  ext4  nofail,noatime,errors=remount-ro  0 0

Ta. I thought it probably was. My Pi server build could not handle 3 USB drives so I need to reconstruct it with just 2, which means cloning the
OS onto a fraction of a larger disk.

Then creating another partition for all the data under /home

Then do a:-

mount -a

---druck

--
"Corbyn talks about equality, justice, opportunity, health care, peace, community, compassion, investment, security, housing...."
"What kind of person is not interested in those things?"

"Jeremy Corbyn?"


--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Fri, 2 Aug 2024 07:05:21 +0100, The Natural Philosopher wrote:

So with the disks attached try

mount | grep '^/dev'

To identify the device name.

lsblk <https://manpages.debian.org/8/lsblk.8.en.html>
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-01 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and sda2
to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

/dev/sda is the WHOLE disk called /dev/sda
/dev/sdaX where X is a number is that PARTITION on the disk /dev/sda

I have not seen anybody mentioning sync
Once the dd command is done, type 'sync'

In some settings the writing is not quite done yet, the sync forces the
write of the caches to the device. I see this on my pc running ubuntu
with 32 Gb RAM.
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 02/08/2024 10:41, Björn Lundin wrote:

On 2024-08-01 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and sda2
to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m


/dev/sda is the WHOLE disk called /dev/sda
/dev/sdaX where X is a number is that PARTITION on the disk /dev/sda

I have not seen anybody mentioning sync
Once the dd command is done, type 'sync'

In some settings the writing is not quite done yet, the sync forces the write of the caches to the device. I see this on my pc running ubuntu
with 32 Gb RAM.

Never mind sync.

It's important to wait anyway on an SSD/nvm until it has finished its
internal business.

For any copy process.
--
All political activity makes complete sense once the proposition that
all government is basically a self-legalising protection racket, is
fully understood.


--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-02 14:51, The Natural Philosopher wrote:

Never mind sync.

It's important to wait anyway on an SSD/nvm until it has finished its internal business.

For any copy process.

dd'ing 500 Gb to an ssd disk, I've seen sync taking 30 s or more
and sync is of course started AFTER dd is done

So - how do you know it is done its internal business?
Not all drives have blinking LEDs

sync makes it easy to know
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 02/08/2024 15:03, Björn Lundin wrote:

On 2024-08-02 14:51, The Natural Philosopher wrote:

Never mind sync.

It's important to wait anyway on an SSD/nvm until it has finished its
internal business.

For any copy process.

dd'ing 500 Gb to an ssd disk, I've seen sync taking 30 s or more
and sync is of course started AFTER dd is done

So - how do you know it is done its internal business?
Not all drives have blinking LEDs

sync makes it easy to know

Even sync may not be enough.
SSDS/NVM have their own internal caching.
--
"In our post-modern world, climate science is not powerful because it is
true: it is true because it is powerful."

Lucas Bergkamp

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-02 16:12, The Natural Philosopher wrote:

On 02/08/2024 15:03, Björn Lundin wrote:

On 2024-08-02 14:51, The Natural Philosopher wrote:

Never mind sync.

It's important to wait anyway on an SSD/nvm until it has finished its
internal business.

For any copy process.

dd'ing 500 Gb to an ssd disk, I've seen sync taking 30 s or more
and sync is of course started AFTER dd is done

So - how do you know it is done its internal business?
Not all drives have blinking LEDs

sync makes it easy to know

Even sync may not be enough.
SSDS/NVM have their own internal  caching.

Then the question remains - how do you know when its done?

from man sync you get
"sync - Synchronize cached writes to persistent storage"

In other words, doing sync at least clear the cache,
which must be better than nothing? or?

Or are you saying that ripping the power to a newly sync()ed disk may
mess it up?

as in

dd ....
sync
rip out usb cord for a 2.5" ssd sata in a usb case or something similar ?
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Fri, 2 Aug 2024 15:12:57 +0100, The Natural Philosopher wrote:

SSDS/NVM have their own internal caching.

True for all drives, unfortunately.

It’s bloody stupid, because the drive caching is on the wrong side of the drive interface. Better to leave it to the OS, which can use main RAM for
its filesystem caching, on the fast side of that drive interface.

When a drive says to the OS driver “write is done”, it should mean “write
has gone to actual persistent storage”, not “write is in my cache”.
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

The Natural Philosopher <tnp@invalid.invalid> writes:

On 02/08/2024 15:03, Björn Lundin wrote:

dd'ing 500 Gb to an ssd disk, I've seen sync taking 30 s or more
and sync is of course started AFTER dd is done
So - how do you know it is done its internal business?
Not all drives have blinking LEDs
sync makes it easy to know

Even sync may not be enough.
SSDS/NVM have their own internal caching.

The sync syscall (and command) will flush those too.

Unless you’ve paid extra for a drive with a huge cache I would expect
the extra delay while the on-drive cache is flushed to be absolutely
tiny in human terms, and certainly tiny compared to flushing the OS’s
cache, which can be multiple gigabytes.

References:
* https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt
* https://linux.die.net/man/2/sync
* https://github.com/torvalds/linux/blob/master/fs/sync.c#L87
--
https://www.greenend.org.uk/rjk/
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 02/08/2024 23:08, Björn Lundin wrote:

On 2024-08-02 16:12, The Natural Philosopher wrote:

On 02/08/2024 15:03, Björn Lundin wrote:

On 2024-08-02 14:51, The Natural Philosopher wrote:

Never mind sync.

It's important to wait anyway on an SSD/nvm until it has finished
its internal business.

For any copy process.

dd'ing 500 Gb to an ssd disk, I've seen sync taking 30 s or more
and sync is of course started AFTER dd is done

So - how do you know it is done its internal business?
Not all drives have blinking LEDs

sync makes it easy to know

Even sync may not be enough.
SSDS/NVM have their own internal  caching.

Then the question remains - how do you know when its done?

from man sync you get
"sync - Synchronize cached writes to persistent storage"

In other words, doing sync at least clear the cache,
which must be better than nothing? or?

Or are you saying that ripping the power to a newly sync()ed disk may
mess it up?

Yes.

as in

dd ....
sync
rip out usb cord for a 2.5" ssd sata in a usb case or something similar ?

I am on the edge of my comfort patch here.
If I were building an SSD I would have a diode and a large capacitor
inside it to make sure all its caches were dumped to NVRAM before the
voltage collapsed completely.

But on a big unit this could take a bit of time.

What happens between a SATA/USB plug and the actual NVRAM is a bit of a mystery.

We know its nothing like a 1:1 correlation between 'sector' and physical
RAM location.
We knows that physical RAM locations are regularly shuffled for 'wear levelling'
When is all this done?
What happens if, during it, there is power failure?

I honestly do not know, hence the warning to leave the SSD for a few
seconds before yanking any power cords

It can do no harm
--
"The most difficult subjects can be explained to the most slow witted
man if he has not formed any idea of them already; but the simplest
thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him."

- Leo Tolstoy


--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-03 12:07, The Natural Philosopher wrote:

I am on the edge of my comfort patch here.
If I were building an SSD I would have a diode and a large capacitor
inside it to make sure all its caches were dumped to NVRAM before the voltage collapsed completely.

But on a big unit this could take a bit of time.

What happens between a SATA/USB plug and the actual NVRAM is a bit of a mystery.

We know its nothing like a 1:1 correlation between 'sector' and physical
RAM location.
We knows that physical RAM locations are regularly shuffled for 'wear levelling'
When is all this done?
What happens if, during it, there is power failure?

I honestly do not know, hence the warning to leave the SSD for a few
seconds before yanking any power cords

It can do no harm

And that is my point. sync will do no harm either,
but it might save you, especially when you are dd()ing an image smaller
that the RAM of the computer.
Like I have 32 Gb RAM, and dd an image of 4 Gb (like a headless) onto a SD-card to run an old pi. The interface is slow, yet dd reports done
within a minute. sync takes a long time to flush it over.

Just waiting a few secs will end in disappointment
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-03 03:27, Lawrence D'Oliveiro wrote:

On Fri, 2 Aug 2024 15:12:57 +0100, The Natural Philosopher wrote:

SSDS/NVM have their own internal caching.

True for all drives, unfortunately.

It’s bloody stupid, because the drive caching is on the wrong side of the drive interface. Better to leave it to the OS, which can use main RAM for
its filesystem caching, on the fast side of that drive interface.

When a drive says to the OS driver “write is done”, it should mean “write
has gone to actual persistent storage”, not “write is in my cache”.

I think they cause grief in the postgres mail lists some 15-20 years ago.
They were called 'lying IDE-disks' and were not popular in that crowd.
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 03/08/2024 12:52, Björn Lundin wrote:

On 2024-08-03 12:07, The Natural Philosopher wrote:

I am on the edge of my comfort patch here.
If I were building an SSD I would have a diode and a large capacitor
inside it to make sure all its caches were dumped to NVRAM before the
voltage collapsed completely.

But on a big unit this could take a bit of time.

What happens between a SATA/USB plug and the actual NVRAM is a bit of
a mystery.

We know its nothing like a 1:1 correlation between 'sector' and
physical RAM location.
We knows that physical RAM locations are regularly shuffled for 'wear
levelling'
When is all this done?
What happens if, during it, there is power failure?

I honestly do not know, hence the warning to leave the SSD for a few
seconds before yanking any power cords

It can do no harm

And that is my point. sync will do no harm either,
but it might save you, especially when you are dd()ing an image smaller
that the RAM of the computer.
Like I have 32 Gb RAM, and dd an image of 4 Gb (like a headless) onto a SD-card to run an old pi. The interface is slow, yet dd reports done
within a minute. sync takes a long time to flush it over.

Just waiting a few secs will end in disappointment

Well I will be at some stage dd-ing about 60GByte across to a 2TB disk,
but given the hassle, sync and a cup of coffee wait before unplugging
it, is likely.

RK claims the wait afterwards isn't necessary. Completing 'sync' is
enough. He generally doesn't comment unless he knows.

I simply do not know how a command to something purporting to resemble a
hard drive, that isnt can guarantee all data in it is flushed.
--
To ban Christmas, simply give turkeys the vote.

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 2024-08-03 13:59, The Natural Philosopher wrote:

I simply do not know how a command to something purporting to resemble a hard drive, that isnt can guarantee all data in it is flushed.

We might talk about different things.
I'm thinking of what the OS is caching. That sync is flushing that part
of the OS cache.

Not whatever clever memory that is inside a storage device.


looking at <https://linux.die.net/man/8/sync>

"""
The kernel keeps data in memory to avoid doing (relatively slow) disk
reads and writes. This improves performance, but if the computer
crashes, data may be lost or the file system corrupted as a result. sync ensures that everything in memory is written to disk.
"""


So, what I think is that dd writes to a slow device, and it is cached by
the OS.
sync forces the OS to actually write to slow device.

This may not be true with new and shiny fast NVM storage, but the
principle holds.

do a sync after a dd operation.
--
/Björn

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

Björn Lundin <bnl@nowhere.com> writes:

On 2024-08-03 03:27, Lawrence D'Oliveiro wrote:

On Fri, 2 Aug 2024 15:12:57 +0100, The Natural Philosopher wrote:

SSDS/NVM have their own internal caching.

True for all drives, unfortunately.
It’s bloody stupid, because the drive caching is on the wrong side
of the
drive interface. Better to leave it to the OS, which can use main RAM for
its filesystem caching, on the fast side of that drive interface.
When a drive says to the OS driver “write is done”, it should mean
“write
has gone to actual persistent storage”, not “write is in my cache”.

I think they cause grief in the postgres mail lists some 15-20 years ago. They were called 'lying IDE-disks' and were not popular in that crowd.

‘Lying disks’ are those that either disregard flush operations, or lie about whether they have a write-back cache at all. That is certainly a
stupid outcome, though as a response to operating systems or
applications that are over-eager to flush, you can see why there’d be pressure from marketing to do it, and acceptance from market segments
that either don’t value data integrity or alternatively assume that the storage is unreliable and address the issue some other way.

I think what Lawrence is complaining about is the fact that the default behavior, even for a non-lying disk, is when a SATA device returns a
response to the host, this may indicate only that data has been
transferred to an internal write-back cache rather than the underlying
medium.

But that’s just the normal engineering response to high physical IO
latency.

Recall that both traditional hard disks and SSDs do not have a 1:1
mapping between the logical block read/writes requested by the host. In
a hard disk it takes time to reach the correct track, and the order of
writes from the host may not match the track order. In an SSD multiple
logical blocks are grouped into pages and pages must be written in a
single operation.

The same logic turns up elsewhere. The write() syscall completing
normally only indicates that data has been transferred to the operating system’s RAM cache, not to your SSD or hard disk (and certainly not to a remote disk, when using a network filesystem). A memory write
instruction on a modern CPU only transfers a value to an internal write
buffer; the data may only reach the external DRAM hundreds of cycles
later, or not at all if the same location is written again soon.

The alternative is absurdly slow write IO. Usually there are some
combination of flush operations, synchronous IO modes, barrier
operations, etc, to allow data integrity requirements to be met without sacrificing performance globally (and this is true both at the Linux
syscall layer and in the SATA protocol ... provided of course your disk
does not lie).
--
https://www.greenend.org.uk/rjk/
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

Bj?rn Lundin <bnl@nowhere.com> wrote:

looking at <https://linux.die.net/man/8/sync>

"""
The kernel keeps data in memory to avoid doing (relatively slow) disk
reads and writes. This improves performance, but if the computer
crashes, data may be lost or the file system corrupted as a result. sync ensures that everything in memory is written to disk.
"""

So, what I think is that dd writes to a slow device, and it is cached by
the OS.
sync forces the OS to actually write to slow device.

I've used the "conv=fsync" option to dd so that sync is done
automatically. Since the man page is rather vague about it, I did
a web search to double check that I wasn't imagining things and
found this page which describes the behaviour with some examples:

https://abbbi.github.io/dd/

This may not be true with new and shiny fast NVM storage, but the
principle holds.

The tests at that link were done with a RAM disk, and the kernel
was caching writes to that, so it doesn't look like the caching
system is smart enough to know when a device is fast enough that
the cache isn't required.
--
__ __
#_ < |\| |< _#
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Sat, 03 Aug 2024 19:51:15 +0100, Richard Kettlewell wrote:

The alternative is absurdly slow write IO.

Caches cannot increase overall transfer bandwidth, they can only reduce latency. Which we don’t care about, because OS filesystem cache flushing
is a batch-type operation anyway.

Which is why having these drives lie about when their writes have
completed is such a dumb idea.
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 04/08/2024 00:30, Computer Nerd Kev wrote:

I've used the "conv=fsync" option to dd so that sync is done
automatically. Since the man page is rather vague about it, I did
a web search to double check that I wasn't imagining things and

Apropos of not too much, I finally got around to copying my old 120GB Pi
boot disk to an image file, and splatting that back onto a 2TB SSD that
now has according to a GUI based disk editor shitloads of 'unused' space.

(The reason I didnt do it directly was on account of a lack of
functional and accessible USB slots in the computer I used.)

All looks ok, with a bootfs and a rootfs partition on it, but I haven't actually booted it yet.

I didn't sync it, I went to the supermarket instead, and left it
running, and then left it running on the second transfer while I had
the evening meal.

It appears to be OK

More later... I overslept badly and it may be Monday before more gets done.
--
“Puritanism: The haunting fear that someone, somewhere, may be happy.”

H.L. Mencken, A Mencken Chrestomathy

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 04/08/2024 13:13, The Natural Philosopher wrote:

On 04/08/2024 00:30, Computer Nerd Kev wrote:

I've used the "conv=fsync" option to dd so that sync is done
automatically. Since the man page is rather vague about it, I did
a web search to double check that I wasn't imagining things and

Apropos of not too much, I finally got around to copying my old 120GB Pi boot disk to an image file, and splatting that back onto a 2TB SSD that
now has according to a GUI based disk editor shitloads of 'unused' space.

(The reason I didnt do it directly was on account of a lack of
functional and accessible USB slots in the computer I used.)

All looks ok, with a bootfs and a rootfs partition on it,  but I haven't actually booted it yet.

I didn't sync it, I went to the supermarket instead, and left it
running, and then left it running on the second transfer while I had the evening meal.

It appears to be OK

More later... I overslept badly and it may be Monday before more gets done.

Tried to reboot the thing and it failed, although it managed to get as
far as system maintenance mode.

Using that mode I then removed all the disks no longer attached, from
fstab, and its now at a normal login prompt!

Next step was to partition the spare space and reattach the second drive

So dd et al work perfectly as far as I can tell. when applied to the raw
disk

I don't know why it barfed on having disks mentioned in fstab that
didn't exist (yet) but it is another thing to be aware of when migrating
a system
--
"When a true genius appears in the world, you may know him by this sign,
that the dunces are all in confederacy against him."

Jonathan Swift.

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Sun, 4 Aug 2024 20:16:16 +0100, The Natural Philosopher wrote:

I don't know why it barfed on having disks mentioned in fstab that
didn't exist (yet) ...

T’was ever thus. Without the “nofail” mount option, the system assumes the
volume absolutely has to be present for a successful boot.
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and sda2
to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from SSD to
nvme.
First there was a complaint about the switch "1m". Changed it to "1b"
and got a complaint about missing permission to open SDA (the SSD I want
to copy from). Threw a sudo at it, and it ran for maybe half an hour,
until it stopped with error "writing nvme0n1, No space left on device".
The SSD and the nvme have the same size, and that seems to be a problem.
Bright ideas are welcome :-)

Below I have copied in what happend in the command line: raspberrypi@raspberrypi:~ $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 238.5G 0 disk
├─sda1 8:1 0 512M 0 part /media/raspberrypi/bootfs
└─sda2 8:2 0 238G 0 part /media/raspberrypi/rootfs
mmcblk0 179:0 0 58G 0 disk
├─mmcblk0p1 179:1 0 512M 0 part /boot/firmware
└─mmcblk0p2 179:2 0 57.5G 0 part /
nvme0n1 259:0 0 238.5G 0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s raspberrypi@raspberrypi:~

Best regards
--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16/08/2024 10:04, Jesper wrote:

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and sda2
to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from SSD to nvme.
First there was a complaint about the switch "1m". Changed it to "1b"
and got a complaint about missing permission to open SDA (the SSD I want
to copy from). Threw a sudo at it, and it ran for maybe half an hour,
until it stopped with error "writing nvme0n1, No space left on device".
The SSD and the nvme have the same size, and that seems to be a problem.
 Bright ideas are welcome :-)

Below I have copied in what happend in the command line: raspberrypi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS sda           8:0    0 238.5G  0 disk
├─sda1        8:1    0   512M  0 part /media/raspberrypi/bootfs
└─sda2        8:2    0   238G  0 part /media/raspberrypi/rootfs
mmcblk0     179:0    0    58G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware └─mmcblk0p2 179:2    0  57.5G  0 part /
nvme0n1     259:0    0 238.5G  0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s raspberrypi@raspberrypi:~

Best regards

WTF is '/nvme0n1' ?

It looks like you have created a file in the root directory of the
device you are copying from....
--
"Nature does not give up the winter because people dislike the cold."

― Confucius

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16.08.2024 12:40, The Natural Philosopher wrote:

On 16/08/2024 10:04, Jesper wrote:

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and
sda2 to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from SSD
to nvme.
First there was a complaint about the switch "1m". Changed it to "1b"
and got a complaint about missing permission to open SDA (the SSD I
want to copy from). Threw a sudo at it, and it ran for maybe half an
hour, until it stopped with error "writing nvme0n1, No space left on
device".
The SSD and the nvme have the same size, and that seems to be a problem.
  Bright ideas are welcome :-)

Below I have copied in what happend in the command line:
raspberrypi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0 238.5G  0 disk
├─sda1        8:1    0   512M  0 part /media/raspberrypi/bootfs
└─sda2        8:2    0   238G  0 part /media/raspberrypi/rootfs
mmcblk0     179:0    0    58G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0  57.5G  0 part /
nvme0n1     259:0    0 238.5G  0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s
raspberrypi@raspberrypi:~

Best regards

Thank you for the reply.

WTF is '/nvme0n1' ?

"nvme0n1" is the "fucking" NVME I have installed in the raspi5.

It looks like you have created a file in the root directory of the
device you are copying from....

I see no trace of the new file you are referring to.

Best regards

--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16/08/2024 12:19, Jesper wrote:

On 16.08.2024 12:40, The Natural Philosopher wrote:

On 16/08/2024 10:04, Jesper wrote:

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and
sda2 to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from SSD
to nvme.
First there was a complaint about the switch "1m". Changed it to "1b"
and got a complaint about missing permission to open SDA (the SSD I
want to copy from). Threw a sudo at it, and it ran for maybe half an
hour, until it stopped with error "writing nvme0n1, No space left on
device".
The SSD and the nvme have the same size, and that seems to be a problem. >>>   Bright ideas are welcome :-)

Below I have copied in what happend in the command line:
raspberrypi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0 238.5G  0 disk
├─sda1        8:1    0   512M  0 part /media/raspberrypi/bootfs
└─sda2        8:2    0   238G  0 part /media/raspberrypi/rootfs
mmcblk0     179:0    0    58G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0  57.5G  0 part /
nvme0n1     259:0    0 238.5G  0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s
raspberrypi@raspberrypi:~

Best regards

Thank you for the reply.

WTF is '/nvme0n1' ?

"nvme0n1" is the "fucking" NVME I have installed in the raspi5.

Not as far as dd is concerned.
What does the outpout of
ls /n*
and
ls /dev/n*
show?

It looks like you have created a file in the root directory of the
device you are copying from....

I see no trace of the new file you are referring to.

Somehow that does not surprise me

Best regards

--
The higher up the mountainside
The greener grows the grass.
The higher up the monkey climbs
The more he shows his arse.

Traditional

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16.08.2024 13:23, The Natural Philosopher wrote:

On 16/08/2024 12:19, Jesper wrote:

On 16.08.2024 12:40, The Natural Philosopher wrote:

On 16/08/2024 10:04, Jesper wrote:

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and
sda2 to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from
SSD to nvme.
First there was a complaint about the switch "1m". Changed it to
"1b" and got a complaint about missing permission to open SDA (the
SSD I want to copy from). Threw a sudo at it, and it ran for maybe
half an hour, until it stopped with error "writing nvme0n1, No space
left on device".
The SSD and the nvme have the same size, and that seems to be a
problem.
  Bright ideas are welcome :-)

Below I have copied in what happend in the command line:
raspberrypi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0 238.5G  0 disk
├─sda1        8:1    0   512M  0 part /media/raspberrypi/bootfs
└─sda2        8:2    0   238G  0 part /media/raspberrypi/rootfs
mmcblk0     179:0    0    58G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0  57.5G  0 part /
nvme0n1     259:0    0 238.5G  0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s
raspberrypi@raspberrypi:~

Best regards

Thank you for the reply.

WTF is '/nvme0n1' ?

"nvme0n1" is the "fucking" NVME I have installed in the raspi5.

Not as far as dd is concerned.
What does the outpout of
ls /n*

raspberrypi@raspberrypi:~ $ ls /n*
ls: klarte ikke å åpne '/n*': Ingen slik fil eller filkatalog
It means "Could not open /n*. No such file or catalog"

and
ls /dev/n*
show?

raspberrypi@raspberrypi:~ $ ls /dev/n*
/dev/ng0n1 /dev/null /dev/nvme0 /dev/nvme0n1

/dev/net:
tun
raspberrypi@raspberrypi:~ $

Should the nvme have been formatted before the copying? I did not do
that because the two partitions on the SSD have different formatting,
one is vfat (boot), the other ext4

Best regards

It looks like you have created a file in the root directory of the
device you are copying from....

I see no trace of the new file you are referring to.

Somehow that does not surprise me

Best regards

--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16.08.2024 13:52, Jesper wrote:

On 16.08.2024 13:23, The Natural Philosopher wrote:

On 16/08/2024 12:19, Jesper wrote:

On 16.08.2024 12:40, The Natural Philosopher wrote:

On 16/08/2024 10:04, Jesper wrote:

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and >>>>>>> sda2 to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from
SSD to nvme.
First there was a complaint about the switch "1m". Changed it to
"1b" and got a complaint about missing permission to open SDA (the
SSD I want to copy from). Threw a sudo at it, and it ran for maybe
half an hour, until it stopped with error "writing nvme0n1, No
space left on device".
The SSD and the nvme have the same size, and that seems to be a
problem.
  Bright ideas are welcome :-)

Below I have copied in what happend in the command line:
raspberrypi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0 238.5G  0 disk
├─sda1        8:1    0   512M  0 part /media/raspberrypi/bootfs
└─sda2        8:2    0   238G  0 part /media/raspberrypi/rootfs
mmcblk0     179:0    0    58G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0  57.5G  0 part /
nvme0n1     259:0    0 238.5G  0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s
raspberrypi@raspberrypi:~

Best regards

Thank you for the reply.

WTF is '/nvme0n1' ?

"nvme0n1" is the "fucking" NVME I have installed in the raspi5.

Not as far as dd is concerned.
What does the outpout of
ls /n*

raspberrypi@raspberrypi:~ $ ls /n*
ls: klarte ikke å åpne '/n*': Ingen slik fil eller filkatalog
It means "Could not open /n*. No such file or catalog"

and
ls /dev/n*
show?

raspberrypi@raspberrypi:~ $ ls /dev/n*
/dev/ng0n1  /dev/null  /dev/nvme0  /dev/nvme0n1

/dev/net:
tun
raspberrypi@raspberrypi:~ $

Should the nvme have been formatted before the copying? I did not do
that because the two partitions on the SSD have different formatting,
one is vfat (boot), the other ext4

Best regards

does this give you any clues:
raspberrypi@raspberrypi:~ $ ls -l /dev/disk/by-uuid
totalt 0
lrwxrwxrwx 1 root root 10 aug. 16 13:53
cb6f0e18-5add-4177-ab98-e9f0235e06b3 -> ../../sda2
lrwxrwxrwx 1 root root 10 aug. 16 13:53 D3E6-3F09 -> ../../sda1
lrwxrwxrwx 1 root root 10 aug. 16 13:53 F40B-6C7F -> ../../sdb1 raspberrypi@raspberrypi:~ $

It looks like you have created a file in the root directory of the
device you are copying from....

I see no trace of the new file you are referring to.

Somehow that does not surprise me

Best regards

--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16/08/2024 12:52, Jesper wrote:

On 16.08.2024 13:23, The Natural Philosopher wrote:

On 16/08/2024 12:19, Jesper wrote:

On 16.08.2024 12:40, The Natural Philosopher wrote:

On 16/08/2024 10:04, Jesper wrote:

On 01.08.2024 21:38, Björn Lundin wrote:

On 2024-08-01 21:02, Jesper wrote:

On 01.08.2024 20:29, Ahem A Rivet's Shot wrote:

On Thu, 1 Aug 2024 19:50:20 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

But I still do not know what a command that clones both sda1 and >>>>>>> sda2 to NVME should look like. Please?

Really? It is given to you more than once

dd if=/dev/sda of=/dev/nvme0n1 bs=1m

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from
SSD to nvme.
First there was a complaint about the switch "1m". Changed it to
"1b" and got a complaint about missing permission to open SDA (the
SSD I want to copy from). Threw a sudo at it, and it ran for maybe
half an hour, until it stopped with error "writing nvme0n1, No
space left on device".
The SSD and the nvme have the same size, and that seems to be a
problem.
  Bright ideas are welcome :-)

Below I have copied in what happend in the command line:
raspberrypi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0 238.5G  0 disk
├─sda1        8:1    0   512M  0 part /media/raspberrypi/bootfs
└─sda2        8:2    0   238G  0 part /media/raspberrypi/rootfs
mmcblk0     179:0    0    58G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0  57.5G  0 part /
nvme0n1     259:0    0 238.5G  0 disk
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1m
dd: invalid number: ‘1m’
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1 bs=1b
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ dd if=/dev/sda of=/nvme0n1
dd: failed to open '/dev/sda': Permission denied
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device
107929249+0 records in
107929248+0 records out
55259774976 bytes (55 GB, 51 GiB) copied, 1476.65 s, 37.4 MB/s
raspberrypi@raspberrypi:~

Best regards

Thank you for the reply.

WTF is '/nvme0n1' ?

"nvme0n1" is the "fucking" NVME I have installed in the raspi5.

Not as far as dd is concerned.
What does the outpout of
ls /n*

raspberrypi@raspberrypi:~ $ ls /n*
ls: klarte ikke å åpne '/n*': Ingen slik fil eller filkatalog
It means "Could not open /n*. No such file or catalog"

So there is no '/nvme0n1' to copy to is there?
IT ISN'T your NVM.

and
ls /dev/n*
show?

raspberrypi@raspberrypi:~ $ ls /dev/n*
/dev/ng0n1  /dev/null  /dev/nvme0  /dev/nvme0n1

Right. so the draw device ce is /dev/nvme0

So try

sudo dd if=/dev/sda of=/dev/nvme0 bs=1M


Note:
/dev/nvme0 NOT /nvme0n1
1M not 1m.





--
“Some people like to travel by train because it combines the slowness of
a car with the cramped public exposure of 
an airplane.”

Dennis Miller


--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Fri, 16 Aug 2024 11:04:26 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device

Your command is missing a /dev it should be:

sudo dd if=/dev/sda of=/dev/nvme0n1 bs=1b
_______________________^^^^^

The "no space on device" error comes because without the /dev dd is attempting to write a file called nvme0n1 in / but since / runs out of
space before it finishes the file is never closed and thus no directory
entry is written which is why yuou never see it.

Also I don't think bs=1b is likely to be a good blocksize (1
byte). If the version of dd you're using doesn't support bs=1m then try bs=1024k or be=1048576. The idea is to get it to use the largest block size (and thus writes) that the data path will support.
--
Steve O'Hara-Smith
Odds and Ends at http://www.sohara.org/
For forms of government let fools contest
Whate're is best administered is best - Alexander Pope
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16/08/2024 13:46, Ahem A Rivet's Shot wrote:

On Fri, 16 Aug 2024 11:04:26 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device

Your command is missing a /dev it should be:

sudo dd if=/dev/sda of=/dev/nvme0n1 bs=1b

No. Again 2 things wroing.

/dev/nvme0 is the RAW device.

bs=1M is correct for decent transfer rate

The "no space on device" error comes because without the /dev dd is attempting to write a file called nvme0n1 in / but since / runs out of
space before it finishes the file is never closed and thus no directory
entry is written which is why yuou never see it.

Also I don't think bs=1b is likely to be a good blocksize (1
byte). If the version of dd you're using doesn't support bs=1m then try bs=1024k or be=1048576. The idea is to get it to use the largest block size (and thus writes) that the data path will support.

bs=1M is sensible.

1Mib.
--
Renewable energy: Expensive solutions that don't work to a problem that doesn't exist instituted by self legalising protection rackets that
don't protect, masquerading as public servants who don't serve the public.


--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16.08.2024 15:17, The Natural Philosopher wrote:

On 16/08/2024 13:46, Ahem A Rivet's Shot wrote:

On Fri, 16 Aug 2024 11:04:26 +0200
Jesper <Vitsky.kasperski@gmail.com> wrote:

raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/nvme0n1 bs=1b
dd: error writing '/nvme0n1': No space left on device

    Your command is missing a /dev it should be:

sudo dd if=/dev/sda of=/dev/nvme0n1 bs=1b

No. Again 2 things wroing.

/dev/nvme0 is the RAW device.

No, it did not like the 'nvmeO'. Here is what it said: raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/nvme0
dd: skriver til '/dev/nvme0': Ugyldig argument
1+0 records in
0+0 records out
0 bytes copied, 0,000298314 s, 0,0 kB/s
'Ugyldig argument' is norwegian for 'illegal argument.
But I managed at last to make the copy like this:
raspberrypi@raspberrypi:~ $ sudo dd if=/dev/sda of=/dev/nvme0n1
500118192+0 records in
500118192+0 records out
256060514304 bytes (256 GB, 238 GiB) copied, 2157,41 s, 119 MB/s raspberrypi@raspberrypi:~ $

After that I shut down the raspi, removed SD-card and SSD. Applied
power, and it booted from the nvme, and I am running on that while
writing this.

Thanks for all patient help from you all :-)

Best regards

bs=1M is correct  for decent transfer rate

    The "no space on device" error comes because without the /dev dd is >> attempting to write a file called nvme0n1 in / but since / runs out of
space before it finishes the file is never closed and thus no directory
entry is written which is why yuou never see it.

    Also I don't think bs=1b is likely to be a good blocksize (1
byte). If the version of dd you're using doesn't support bs=1m then try
bs=1024k or be=1048576. The idea is to get it to use the largest block
size
(and thus writes) that the data path will support.

bs=1M is sensible.

 1Mib.

--
Jesper

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 16/08/2024 10:04, Jesper wrote:

Now I have the nvme installed and showing in a lsblk command.
Booted from a SD-card and did a few tries to copy the system from SSD to nvme.
First there was a complaint about the switch "1m". Changed it to "1b"

Arrh, that's asking to copy a single 512 byte block at a time - horribly
in efficient. Try "1M" to copy in 1 MiB chunks.

and got a complaint about missing permission to open SDA (the SSD I want
to copy from). Threw a sudo at it,

Yes you need root permission to write to block devices, given how easy
it is to destroy the filing system on them with an even a slightly
incorrect command.

and it ran for maybe half an hour,
until it stopped with error "writing nvme0n1, No space left on device".
The SSD and the nvme have the same size, and that seems to be a problem.
 Bright ideas are welcome :-)

But are they? Just like with SD cards, no two models are exactly the
same size. If it is even a block less dd will complain, check the sizes using:-

sudo fdisk -l


If it is smaller it can be fixed using the gparted program. Calculate
the size difference in bytes and divide down to MB, then run gparted and
go to source disc (/dev/sda) and resize the last partition (rootfs) to
be X+1 MB smaller. Finish by apply the changes.

You can then do the dd copy again, it will still give the error, but the
copy will be good as it is now only empty space which wont have been
copied over.

You might be asking why don't you just use gparted on the destination
disk which already has everything on it, and shrink the last partition
on that. The answer is it wont like doing anything to a disc where the partition table says the partition falls off the end of the disc. It is possible to fix that, but it's easier to do it the way I've said above,
and it avoids any possibility of corrupting files.

---druck

--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On Fri, 16 Aug 2024 21:34:42 +0100, druck wrote:

On 16/08/2024 10:04, Jesper wrote:

The SSD and the nvme have the same size ...

But are they?

This is why rsync is so much easier.
--- Synchronet 3.19b-Win32 NewsLink 1.113

From Newsgroup: comp.sys.raspberry-pi

On 17/08/2024 06:45, Lawrence D'Oliveiro wrote:

On Fri, 16 Aug 2024 21:34:42 +0100, druck wrote:

On 16/08/2024 10:04, Jesper wrote:

The SSD and the nvme have the same size ...

But are they?

This is why rsync is so much easier.

It isn't, but my original post in this thread gave full instructions on
how to do that too.

---druck
--- Synchronet 3.19b-Win32 NewsLink 1.113