The VMDK is on the host, on an NVME SSD.
it shouldn’t be that bad – no way – no how.
I need check something here.
Sure. sounds good.
I will post the specs of the baremetal in this post via an edit. One second.
(t10.NVMe____Samsung_SSD_970_PRO_1TB_________________EA43B39156382500)
If I recall correctly, it was an X470 ASRock motherboard.
smbiosDump |grep -A 5 'System Info'
System Info (Type 1): #1
Manufacturer: "To Be Filled By O.E.M."
Product: "To Be Filled By O.E.M."
Version: "To Be Filled By O.E.M."
Serial: "To Be Filled By O.E.M."
SKU: "To Be Filled By O.E.M."
smbiosDump |grep -A 20 'Processor Info'
Processor Info (Type 4): #21
Payload length: 0x30
Socket: "AM4"
Socket Type: 0x31 (Socket AM4)
Socket Status: Populated
Type: 0x03 (CPU)
Family: 0x6b (Zen)
Manufacturer: "Advanced Micro Devices, Inc."
Version: "AMD Ryzen 7 1700 Eight-Core Processor"
Serial: "Unknown"
Asset Tag: "Unknown"
Part Number: "Unknown"
Processor ID: 0x178bfbff00800f11
Status: 0x01 (Enabled)
Voltage: 1.2 V
External Clock: 100 MHz
Max. Speed: 3750 MHz
Current Speed: 3000 MHz
L1 Cache: #18
L2 Cache: #19
L3 Cache: #20
Memory Device (Type 17): #23
Location: "DIMM 0"
Bank: "P0 CHANNEL A"
Manufacturer: "Micron Technology"
Serial: "1B819B15"
Part Number: "18ASF2G72AZ-2G3B1"
Memory Array: #16
Error Info: #22
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x4080 (Synchronous, Unregistered)
Data Width: 64 bits (+64 ECC bits)
Size: 16 GB
--
Memory Device Mapping (Type 20): #24
Memory Device: #23
Array Mapping: #17
Start Address: 0x0000000000000000
End Address: 0x0000001000000000
32bit-Memory Error Info (Type 18): #25
Type: 0x03 (OK)
Granularity: 0x02 (Unknown)
Operation: 0x02 (Unknown)
Memory Device (Type 17): #26
Location: "DIMM 1"
Bank: "P0 CHANNEL A"
Manufacturer: "Micron Technology"
Serial: "1B819694"
Part Number: "18ASF2G72AZ-2G3B1"
Memory Array: #16
Error Info: #25
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x4080 (Synchronous, Unregistered)
Data Width: 64 bits (+64 ECC bits)
Size: 16 GB
--
Memory Device Mapping (Type 20): #27
Memory Device: #26
Array Mapping: #17
Start Address: 0x0000000000000000
End Address: 0x0000001000000000
32bit-Memory Error Info (Type 18): #28
Type: 0x03 (OK)
Granularity: 0x02 (Unknown)
Operation: 0x02 (Unknown)
Memory Device (Type 17): #29
Location: "DIMM 0"
Bank: "P0 CHANNEL B"
Manufacturer: "Micron Technology"
Serial: "1B819C68"
Part Number: "18ASF2G72AZ-2G3B1"
Memory Array: #16
Error Info: #28
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x4080 (Synchronous, Unregistered)
Data Width: 64 bits (+64 ECC bits)
Size: 16 GB
--
Memory Device Mapping (Type 20): #30
Memory Device: #29
Array Mapping: #17
Start Address: 0x0000000000000000
End Address: 0x0000001000000000
32bit-Memory Error Info (Type 18): #31
Type: 0x03 (OK)
Granularity: 0x02 (Unknown)
Operation: 0x02 (Unknown)
Memory Device (Type 17): #32
Location: "DIMM 1"
Bank: "P0 CHANNEL B"
Manufacturer: "Micron Technology"
Serial: "1B819FDD"
Part Number: "18ASF2G72AZ-2G3B1"
Memory Array: #16
Error Info: #31
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x4080 (Synchronous, Unregistered)
Data Width: 64 bits (+64 ECC bits)
Size: 16 GB
--
Memory Device Mapping (Type 20): #33
Memory Device: #32
Array Mapping: #17
Start Address: 0x0000000000000000
End Address: 0x0000001000000000
smbiosDump |grep -A 4 'Physical Memory Array'
Physical Memory Array (Type 16): #16
Use: 0x03 (System memory)
Location: 0x03 (Motherboard)
Slots: 4
Max. Size: 128 GB
[root@esxi2:~] smbiosDump |grep -A 5 'System Info'
System Info (Type 1): #1
Manufacturer: "To Be Filled By O.E.M."
Product: "To Be Filled By O.E.M."
Version: "To Be Filled By O.E.M."
Serial: "To Be Filled By O.E.M."
SKU: "To Be Filled By O.E.M."
[root@esxi2:~] smbiosDump |grep -A 20 'Processor Info'
Processor Info (Type 4): #21
Payload length: 0x30
Socket: "AM4"
Socket Type: 0x31 (Socket AM4)
Socket Status: Populated
Type: 0x03 (CPU)
Family: 0x6b (Zen)
Manufacturer: "Advanced Micro Devices, Inc."
Version: "AMD Ryzen 7 1700 Eight-Core Processor"
Serial: "Unknown"
Asset Tag: "Unknown"
Part Number: "Unknown"
Processor ID: 0x178bfbff00800f11
Status: 0x01 (Enabled)
Voltage: 1.2 V
External Clock: 100 MHz
Max. Speed: 3750 MHz
Current Speed: 3000 MHz
L1 Cache: #18
L2 Cache: #19
L3 Cache: #20
[root@esxi2:~] smbiosDump |grep -A 11 'Cache Info'
Cache Info (Type 7): #18
Designation: "L1 - Cache"
Level: L1
State: Enabled
Mode: 0x01 (Write Back)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x06 (Multi-bit)
Type: 0x05 (Unified)
Associativity: 0x07 (8-way Set-Associative)
Max. Size: 768 kB
Current Size: 768 kB
Speed: 1 ns
--
Cache Info (Type 7): #19
Designation: "L2 - Cache"
Level: L2
State: Enabled
Mode: 0x01 (Write Back)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x06 (Multi-bit)
Type: 0x05 (Unified)
Associativity: 0x07 (8-way Set-Associative)
Max. Size: 4096 kB
Current Size: 4096 kB
Speed: 1 ns
--
Cache Info (Type 7): #20
Designation: "L3 - Cache"
Level: L3
State: Enabled
Mode: 0x01 (Write Back)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x06 (Multi-bit)
Type: 0x05 (Unified)
Associativity: 0x08 (16-way Set-Associative)
Max. Size: 16384 kB
Current Size: 16384 kB
Speed: 1 ns
While I am not going to wipe out the existing drive…
@apollo:/mnt/plex$ sudo dd if=/dev/urandom of=/mnt/plex/testfile.fake bs=1M count=105MB
[sudo] password for :
^C11117+0 records in
11116+0 records out
11655970816 bytes (12 GB, 11 GiB) copied, 183.181 s, 63.6 MB/s
For reference.
Ubuntu 19.10, VMware VM, 4GB , 1 CPU, 8 core, 64 GB vmdk. – over the LAN.
Monolithic VMDK or fragmented? This VMDK is monolithic
I gave you the answer in the edit, on the post above yours.
thanks.
Since you’re local and mine is running, effectively, iSCSI over the LAN, your local NMVE should be smoking at 2.7 GB/sec nomimally. The I/O stats don’t show that.
What does a simple dd like I did show?
I created a brand new vmdk and tested, and then also tested against the existing vmdk. Both of these vmdk exist on the same NVME SSD. sda is the existing, sdd is the new one:
andrew@apollo:~$ sudo fdisk /dev/sdd
Welcome to fdisk (util-linux 2.34).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xe423a3e4.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p):
Using default response p.
Partition number (1-4, default 1):
First sector (2048-134217727, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-134217727, default 134217727):
Created a new partition 1 of type 'Linux' and of size 64 GiB.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
andrew@apollo:~$ sudo mkfs
mkfs mkfs.bfs mkfs.btrfs mkfs.cramfs mkfs.ext2 mkfs.ext3 mkfs.ext4 mkfs.fat mkfs.minix mkfs.msdos mkfs.ntfs mkfs.vfat mkfs.xfs
andrew@apollo:~$ sudo mkfs
mkfs mkfs.bfs mkfs.btrfs mkfs.cramfs mkfs.ext2 mkfs.ext3 mkfs.ext4 mkfs.fat mkfs.minix mkfs.msdos mkfs.ntfs mkfs.vfat mkfs.xfs
andrew@apollo:~$ sudo mkfs.ext4 /dev/sdd
mke2fs 1.45.5 (07-Jan-2020)
Found a dos partition table in /dev/sdd
Proceed anyway? (y,N) y
Creating filesystem with 16777216 4k blocks and 4194304 inodes
Filesystem UUID: 500830b1-a52c-4aba-ad7f-c4b012d7278d
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424
Allocating group tables: done
Writing inode tables: done
Creating journal (131072 blocks): done
Writing superblocks and filesystem accounting information: done
andrew@apollo:~$ sudo dd if=/dev/sdd of=/dev/null bs=1M
65536+0 records in
65536+0 records out
68719476736 bytes (69 GB, 64 GiB) copied, 19.0623 s, 3.6 GB/s
andrew@apollo:~$ sudo dd if=/dev/sda of=/dev/null bs=1M
24576+0 records in
24576+0 records out
25769803776 bytes (26 GB, 24 GiB) copied, 29.965 s, 860 MB/s
I also tried re-running after a few minutes of uptime, and it still came back with roughly the same results.
Try it with ext4. I use ext4. BTRFS adds an additional layer of overhead.
I am using ext4 here, btrfs is not in use here. All filesystems on this machine are ext4.
$ mount | grep -i 'dev/sd'
/dev/sda2 on / type ext4 (rw,relatime)
/dev/sdb1 on /mnt/plex type ext4 (rw,noatime,nodiratime,nobarrier,errors=remount-ro)
Let me read through this carefully again. it’s right here, whatever it is
Oh (****)… I just realized I was looking direct at your response…
One second… /dev/sdb1 is the plex partition. Either way, they are all on the same NVME, but just for completion of all facts:
apollo:~$ sudo dd if=/dev/sdb of=/dev/null bs=1M
204800+0 records in
204800+0 records out
214748364800 bytes (215 GB, 200 GiB) copied, 197.776 s, 1.1 GB/s
The raw VMDK should be full device speed.
Yeah, but I just want to make sure that everything there is the completion of all data. I updated my previous post with the dd command from the plex drive. It is still more than fast enough for this operation I would think.
it’s not so much the raw read/write speed (but that’s where it starts) it’s the transactions/sec which we can get out of it.
I am going to test some more there. I am looking at it but not seeing it. (forest before the trees)
This is my NUC8.
[chuck@lizum ~.502]$ sudo dd if=/dev/nvme0n1 of=/dev/null bs=1M count=50000
50000+0 records in
50000+0 records out
52428800000 bytes (52 GB, 49 GiB) copied, 19.051 s, 2.8 GB/s
[chuck@lizum ~.503]$
This is a config issue somewhere.
Time to rethink.
When you say config issue, you mean:
Docker Container, NVME drive, esxi, etc?
Or configuration issue with the libraries? Can you expand more on that?
I can’t run the dd command directly against the nvme disk at the esxi level, and the VMs are looking at as an ssd disk, not nvme device. So I can’t provide any context there.
Configuration with the composite configuration.
Docker + Guest + ESXi VM
For laughs, which you can easily remove, download and install PMS native.
Don’t install a SNAP
Doing so will remove Docker as a variable or point to Docker
Then is there a tool to utilize to determine generic transactions per second?
Never done a transactions per second test before, but if you think that is the case, I feel like that might be a good test. So far from what I have researched, it seems like sysbench or iozone3 might do this?
hdparm -Tt /dev/devi
[chuck@lizum ~.504]$ hdparm -Tt /dev/nvme0n1
/dev/nvme0n1: Permission denied
[chuck@lizum ~.505]$ sudo !!
sudo hdparm -Tt /dev/nvme0n1
/dev/nvme0n1:
Timing cached reads: 32718 MB in 1.99 seconds = 16463.74 MB/sec
HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
Timing buffered disk reads: 8246 MB in 3.00 seconds = 2748.11 MB/sec
[chuck@lizum ~.506]$
