Addonics SIS3124 Controller and T

[Jose Amengual]

Hi guys.

I just recently bought this controller :

http://www.addonics.com/products/host_controller/adsa3gpx8-4e.asp

Plus :

http://www.addonics.com/products/raid_tower/raid_rack.asp ( the one
with 4esata connectors in the back)

so I have a 8.1-RELEASE running and the controlled connected and the
siis driver loaded plus 10 HDs installed on the raidrack each 5 is
connected to a 5x1 port mutiplier and then to a esata.

so here is the issue :

1.- I can see just only from ada0 to ada6 I don't see any other HDs.

2.- Performance is just 131 MBs per second as report from diskinfo -t

superstorage1# diskinfo -t /dev/ada0
/dev/ada0
512 # sectorsize
1000204886016 # mediasize in bytes (932G)
1953525168 # mediasize in sectors
0 # stripesize
0 # stripeoffset
1938021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
WD-WCATR5253676 # Disk ident.

Seek times:
Full stroke: 250 iter in 5.055337 sec = 20.221 msec
Half stroke: 250 iter in 3.685852 sec = 14.743 msec
Quarter stroke: 500 iter in 5.924198 sec = 11.848 msec
Short forward: 400 iter in 2.157650 sec = 5.394 msec
Short backward: 400 iter in 1.785152 sec = 4.463 msec
Seq outer: 2048 iter in 0.292870 sec = 0.143 msec
Seq inner: 2048 iter in 0.232126 sec = 0.113 msec
Transfer rates:
outside: 102400 kbytes in 0.771598 sec = 132712 kbytes/sec
middle: 102400 kbytes in 0.925421 sec = 110652 kbytes/sec
inside: 102400 kbytes in 1.652984 sec = 61949 kbytes/sec

it should be more ?

3.- I have two different system with same controller one is a dell
2850 and the other one is a supermicro server in both I can see only 7
HDs.

Could be a problem with the driver ?

So my idea was that to use this with ZFS and AOE so I can have a SAN
for my servers but I need to get the best performance possible.

I'm willing to buy another controller if is necessary, any
recommendations ? ( no more than 700 US )

some additional info :

***@pci0:9:0:0: class=0x018000 card=0x31241095 chip=0x31241095
rev=0x02 hdr=0x00
vendor = 'Silicon Image Inc (Was: CMD Technology Inc)'
device = 'PCI-X to Serial ATA Controller (SiI 3124)'
class = mass storage

***@pci0:7:0:0: class=0x010400 card=0x02ca15d9 chip=0x02859005
rev=0x09 hdr=0x00
vendor = 'Adaptec Inc'
device = 'PCIX133 32/64bit (Adaptec 2410SA SATA RAID)'
class = mass storage
subclass = RAID


Thanks.

[d***@engineer.com]

Post by Jose Amengual
1.- I can see just only from ada0 to ada6 I don't see any other HDs.

Are both of the JMB393s configured as individual drives port multiplier
mode? (as opposed to some hardware raid mode) I assume you've already
checked the cables.

Post by Jose Amengual
2.- Performance is just 131 MBs per second as report from diskinfo -t

This is for an individual drive, correct? How fast do you expect it to
be?

Have you tried connecting individual drives to the 3124 controller
and tested the performance that way?

Post by Jose Amengual
From what you've written, the bottleneck could be the controller, the

port multiplier, or the drive itself. 131 MB/s sounds about right
for a single drive, unless it is some super high performance drive
or you are reading from the drive's RAM rather than from the platters.
I can get 253 MB/s reading from a drive's RAM. Useful for seeing what
controllers, port multipliers and such can do without the limits of
the platter density. (the platter density goes up as new generations
of drives come out) I'd be interested in knowing what the 3124
controller and the JMB393 PM can do.

[Jose Amengual]

Post by d***@engineer.com

Post by Jose Amengual
1.- I can see just only from ada0 to ada6 I don't see any other HDs.

Are both of the JMB393s configured as individual drives port multiplier
mode? (as opposed to some hardware raid mode) I assume you've already
checked the cables.

Strangely enough just switching drives now I can see all ten, weird but is working now.

I'm confused why you are referring to a JMB393s if my card has a siis3124 chip....or are you referring o the PMs?

Post by d***@engineer.com

Post by Jose Amengual
2.- Performance is just 131 MBs per second as report from diskinfo -t

This is for an individual drive, correct?
How fast do you expect it to
be?

This was for an individual drive.

I was expecting more but it looks like I was wrong with my assumptions.

Post by d***@engineer.com
Have you tried connecting individual drives to the 3124 controller
and tested the performance that way?

I didn't try this, I will do that.

Post by d***@engineer.com

Post by Jose Amengual
From what you've written, the bottleneck could be the controller, the

port multiplier, or the drive itself. 131 MB/s sounds about right
for a single drive, unless it is some super high performance drive
or you are reading from the drive's RAM rather than from the platters.
I can get 253 MB/s reading from a drive's RAM. Useful for seeing what
controllers, port multipliers and such can do without the limits of
the platter density. (the platter density goes up as new generations
of drives come out) I'd be interested in knowing what the 3124
controller and the JMB393 PM can do.
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-hardware

[d***@engineer.com]

Post by Jose Amengual
Strangely enough just switching drives now I can see all ten, weird
but is working now.

Perhaps there was a loose conection or something.

Post by Jose Amengual
I'm confused why you are referring to a JMB393s if my card has a
siis3124 chip....or are you referring o the PMs?

If I'm reading
http://www.addonics.com/products/raid_tower/raid_rack.asp
correctly, it comes with 4 SATA Port Multipliers based on
the JMB393 chip.

Do you get something like

PM Product ID: 37261095

in dmesg? I just got a PM based on the Sil3726. So far so good.
I've been running some simple speed tests on it and various
controllers. If you and your machine have a few minutes to spare,
I'd love to see similar numbers for the siis3124 controller and the
JMB393 PM.

First I ran one disk at a time:

nforce4-ultra controller chipset ata(4)
ad4 249.404517 MB/s
ad6 250.143419 MB/s
ad8 239.214086 MB/s
ad10 249.628334 MB/s

SiI 3132 PCIe-x1 siis(4)
ada0 127.737664 MB/s
ada1 110.236024 MB/s
ada2 110.210423 MB/s

JMB363 PCIe-x1 ahci(4)
ada3 131.443644 MB/s
Sil3726 Port Multiplier
ada4 130.895997 MB/s
ada5 130.259701 MB/s
ada6 132.400908 MB/s
ada7 114.306864 MB/s
ada8 130.986963 MB/s

I need to rerun the tests (machine is busy right now) and see if the
results are consistant. Some of the differences may be due to the
different make & model drives.

2 disks at once - port multiplier drives ada4, ada5
48.034694 MB/s and 48.034701 MB/s
Total: 96.069395 MB/s

2 disks at once - same card (jmb363) ada3 direct connection, ada5 via PM
86.059754 MB/s and 85.664682 MB/s
Total: 171.724436 MB/s

3 disks at once - port multiplier drives ada4, ada5, ada6
31.728884 MB/s, 31.727878 MB/s, and 31.727715 MB/s
Total: 95.184477 MB/s

4 disks at once - port multiplier drives ada4, ada5, ada6, ada8
23.657665 MB/s 23.657404 MB/s 23.657416 MB/s 23.657436 MB/s
Total: 94.629921 MB/s

5 port multiplier disks at once
18.660235 MB/s 18.659920 MB/s 18.659928 MB/s 18.659925 MB/s
18.665027 MB/s
5 port multiplier disks at once plus the other port of the jmb363
105.348778 MB/s 17.821848 MB/s 17.821214 MB/s 17.821113 MB/s
17.821120 MB/s 17.821458 MB/s
PM total (5 drives): 89.106753 MB/s
Card total (6 drives): 194.455531 MB/s

Neither the 3132 nor the 363 controllers can max out the PCIe-x1 link
(250 MB/s each direction). Previous testing indicates that both
achive a higher total with one disk reading and the other writing.
(not sure why) Port multipliers may be the same, I haven't had time
to test that case yet. I also want to test the PM with the 3132.

Silicon Image SATA controllers are not the fastest. The JMB363 is
faster than the 3132, so I wonder if the JMB393 port multiplier is
faster than the Sil3726? I've read that the 3124 controller is
fast, the question is how fast?

Below is the small C program I used. It just reads a small amount
of data over and over again from the drive's RAM cache. This avoids
limitations of data density on the platters, rpm, seek time, etc.


/*
* read_disk_cache_speed.c
*
* Read data from a disk's RAM cache buffer.
* Useful for testing the speed of controllers and port multipliers
* without the limitation of how fast the disk can read data from the
* platters. Note that the drive's electronics could still be the
* limiting factor.
*
* Syntax: read_disk_cache_speed device readsize iterations
* FreeBSD:
* read_disk_cache_speed /dev/ada0 65536 10000
* NetBSD:
* read_disk_cache_speed /dev/rwd0c 65536 10000
*
* Original program was by:
* Manuel Bouyer <***@antioche.eu.org>
* NetBSD: 26 ans d'experience feront toujours la difference
* Updated by:
* Dieter <***@engineer.com>
* Added command line argument for buffer/read size.
* (originally hard coded at 64 KiB)
* Read data into drive's RAM buffer before beginning timing.
*
*/

#include <fcntl.h>
#include <unistd.h>
#include <sys/time.h>
#include <stdio.h>
#include <stdlib.h>

int
main(int argc, char **argv)
{
int fd; /* file descriptor */
int i; /* loop counter */
struct timeval tv0, tv1;
long t; /* elapsed time */
char *buffer;
long buffer_size;

if ( argc != 4 )
{
fprintf(stderr, "Syntax: %s device readsize iterations\n",
argv[0]);
fprintf(stderr, "Example: %s /dev/ada4 65536 10000\n", argv[0]);
exit(1);
}

buffer_size = atoi(argv[2]);
buffer = malloc(buffer_size);
if ( buffer == NULL )
{
perror("malloc failed");
exit(1);
}

fd = open(argv[1], O_RDONLY, 0);
if (fd < 0)
{
perror("open failed");
exit(1);
}
/* Get the data from the platter into the drive's RAM buffer
* before we start timing.
*/
if (read(fd, (void *)buffer, buffer_size) != buffer_size)
{
perror("read failed");
exit(1);
}
if (lseek(fd, 0, SEEK_SET) < 0)
{
perror("lseek failed");
exit(1);
}

if (gettimeofday(&tv0, NULL) < 0)
{
perror("gettimeofday failed");
exit(1);
}
for (i = 0; i < atoi(argv[3]); i++)
{
if (read(fd, (void *)buffer, buffer_size) != buffer_size)
{
perror("read failed");
exit(1);
}
if (lseek(fd, 0, SEEK_SET) < 0)
{
perror("lseek failed");
exit(1);
}
}
if (gettimeofday(&tv1, NULL) < 0)
{
perror("gettimeofday failed");
exit(1);
}
t = (tv1.tv_sec - tv0.tv_sec) * 1000000;
t = t + tv1.tv_usec - tv0.tv_usec;
printf("%ld us, %f MB/s\n", t,
((double)buffer_size * (double)i / 1000000) / ((double)t / 1000000));
exit(0);
}

Total: 93.305028 MB/s

[Jose Amengual]

Post by Jose Amengual
Strangely enough just switching drives now I can see all ten, weird
but is working now.

Perhaps there was a loose connection or something.

I think so

Post by Jose Amengual
I'm confused why you are referring to a JMB393s if my card has a
siis3124 chip....or are you referring o the PMs?

If I'm reading
http://www.addonics.com/products/raid_tower/raid_rack.asp
correctly, it comes with 4 SATA Port Multipliers based on
the JMB393 chip.

I see....

Do you get something like
PM Product ID: 37261095

I will check

in dmesg? I just got a PM based on the Sil3726. So far so good.
I've been running some simple speed tests on it and various
controllers. If you and your machine have a few minutes to spare,
I'd love to see similar numbers for the siis3124 controller and the
JMB393 PM.
nforce4-ultra controller chipset ata(4)
ad4 249.404517 MB/s
ad6 250.143419 MB/s
ad8 239.214086 MB/s
ad10 249.628334 MB/s
SiI 3132 PCIe-x1 siis(4)
ada0 127.737664 MB/s
ada1 110.236024 MB/s
ada2 110.210423 MB/s
JMB363 PCIe-x1 ahci(4)
ada3 131.443644 MB/s
Sil3726 Port Multiplier
ada4 130.895997 MB/s
ada5 130.259701 MB/s
ada6 132.400908 MB/s
ada7 114.306864 MB/s
ada8 130.986963 MB/s
I need to rerun the tests (machine is busy right now) and see if the
results are consistant. Some of the differences may be due to the
different make & model drives.
2 disks at once - port multiplier drives ada4, ada5
48.034694 MB/s and 48.034701 MB/s
Total: 96.069395 MB/s
2 disks at once - same card (jmb363) ada3 direct connection, ada5 via PM
86.059754 MB/s and 85.664682 MB/s
Total: 171.724436 MB/s
3 disks at once - port multiplier drives ada4, ada5, ada6
31.728884 MB/s, 31.727878 MB/s, and 31.727715 MB/s
Total: 95.184477 MB/s
4 disks at once - port multiplier drives ada4, ada5, ada6, ada8
23.657665 MB/s 23.657404 MB/s 23.657416 MB/s 23.657436 MB/s
Total: 94.629921 MB/s
5 port multiplier disks at once
18.660235 MB/s 18.659920 MB/s 18.659928 MB/s 18.659925 MB/s
18.665027 MB/s
5 port multiplier disks at once plus the other port of the jmb363
105.348778 MB/s 17.821848 MB/s 17.821214 MB/s 17.821113 MB/s
17.821120 MB/s 17.821458 MB/s
PM total (5 drives): 89.106753 MB/s
Card total (6 drives): 194.455531 MB/s
Neither the 3132 nor the 363 controllers can max out the PCIe-x1 link
(250 MB/s each direction). Previous testing indicates that both
achive a higher total with one disk reading and the other writing.
(not sure why) Port multipliers may be the same, I haven't had time
to test that case yet. I also want to test the PM with the 3132.
Silicon Image SATA controllers are not the fastest. The JMB363 is
faster than the 3132, so I wonder if the JMB393 port multiplier is
faster than the Sil3726? I've read that the 3124 controller is
fast, the question is how fast?
Below is the small C program I used. It just reads a small amount
of data over and over again from the drive's RAM cache. This avoids
limitations of data density on the platters, rpm, seek time, etc.

I check the raid controler and it doesn't have anything to enable any level of caching but I have a Supermicro sata controller with cache memory and and Intel chip that I will use for some testings.

/*
* read_disk_cache_speed.c
*
* Read data from a disk's RAM cache buffer.
* Useful for testing the speed of controllers and port multipliers
* without the limitation of how fast the disk can read data from the
* platters. Note that the drive's electronics could still be the
* limiting factor.
*
* Syntax: read_disk_cache_speed device readsize iterations
* read_disk_cache_speed /dev/ada0 65536 10000
* read_disk_cache_speed /dev/rwd0c 65536 10000
*
* NetBSD: 26 ans d'experience feront toujours la difference
* Added command line argument for buffer/read size.
* (originally hard coded at 64 KiB)
* Read data into drive's RAM buffer before beginning timing.
*
*/
#include <fcntl.h>
#include <unistd.h>
#include <sys/time.h>
#include <stdio.h>
#include <stdlib.h>
int
main(int argc, char **argv)
{
int fd; /* file descriptor */
int i; /* loop counter */
struct timeval tv0, tv1;
long t; /* elapsed time */
char *buffer;
long buffer_size;
if ( argc != 4 )
{
fprintf(stderr, "Syntax: %s device readsize iterations\n",
argv[0]);
fprintf(stderr, "Example: %s /dev/ada4 65536 10000\n", argv[0]);
exit(1);
}
buffer_size = atoi(argv[2]);
buffer = malloc(buffer_size);
if ( buffer == NULL )
{
perror("malloc failed");
exit(1);
}
fd = open(argv[1], O_RDONLY, 0);
if (fd < 0)
{
perror("open failed");
exit(1);
}
/* Get the data from the platter into the drive's RAM buffer
* before we start timing.
*/
if (read(fd, (void *)buffer, buffer_size) != buffer_size)
{
perror("read failed");
exit(1);
}
if (lseek(fd, 0, SEEK_SET) < 0)
{
perror("lseek failed");
exit(1);
}
if (gettimeofday(&tv0, NULL) < 0)
{
perror("gettimeofday failed");
exit(1);
}
for (i = 0; i < atoi(argv[3]); i++)
{
if (read(fd, (void *)buffer, buffer_size) != buffer_size)
{
perror("read failed");
exit(1);
}
if (lseek(fd, 0, SEEK_SET) < 0)
{
perror("lseek failed");
exit(1);
}
}
if (gettimeofday(&tv1, NULL) < 0)
{
perror("gettimeofday failed");
exit(1);
}
t = (tv1.tv_sec - tv0.tv_sec) * 1000000;
t = t + tv1.tv_usec - tv0.tv_usec;
printf("%ld us, %f MB/s\n", t,
((double)buffer_size * (double)i / 1000000) / ((double)t / 1000000));
exit(0);
}

I will use your program to test my hds and I will let you know.

Total: 93.305028 MB/s
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-hardware

[jamengual]

After a lot of time doing some stuff here are the results on the sis
controller and a interna supermicro controller I don't know why the internal
supermicro controller is so fast.

Here are the results:

The following results are from testing from the drives connected via port
multiplier to the RAID DSA3GPX8-4E controller. Note ada0, ada1, ada2, ada3
and ada4 did not have the SATA2 jumpers, while the rest of the drives did.

[***@superstorage ~]# ./read_disk_cache_speed /dev/ada0 65536 10000
10009639 us, 114.472891 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada1 65536 10000
5713195 us, 114.709895 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada2 65536 10000
5702896 us, 114.917053 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada3 65536 10000
5719617 us, 114.581099 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada4 65536 10000
5720035 us, 114.572726 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada5 65536 10000
3435868 us, 190.740739 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada6 65536 10000
3593060 us, 182.396064 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada7 65536 10000
3490376 us, 187.762006 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada8 65536 10000
3491744 us, 187.688445 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/ada9 65536 10000
3491041 us, 187.726240 MB/s

Direct connection from the drive to the card yielded the same results as
above.

====================================================================

These results were from the SuperMicro SMC AOC-USAS-S8iR controller. With
read cache disabled, the speed is 7.89 MB/s while read cache enabled allows
speeds at around 600MB/s

***@superstorage ~]# arcconf setcache 1 logicaldrive 1 roff
Controllers found: 1

Command completed successfully.
[***@superstorage ~]# ./read_disk_cache_speed /dev/aacd1 65536 10000
83328224 us, 7.864802 MB/s

[***@superstorage ~]# arcconf setcache 1 logicaldrive 1 ron
Controllers found: 1

Command completed successfully.
[***@superstorage ~]# ./read_disk_cache_speed /dev/aacd1 65536 10000
1107666 us, 591.658496 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/aacd1 65536 10000
1048647 us, 624.957684 MB/s
[***@superstorage ~]# ./read_disk_cache_speed /dev/aacd1 65536 10000
1048619 us, 624.974371 MB/s

I'm suspecting that the supermicro controller has a bigger cache memory
therefore the performance increase.

I need to thank Sum my "helper" on getting this testing done for me.

Thanks.

--
View this message in context: http://freebsd.1045724.n5.nabble.com/Addonics-SIS3124-Controller-and-T-tp4220441p4390480.html
Sent from the freebsd-hardware mailing list archive at Nabble.com.