[osdcmy] LUN from SAN storage got corrupted

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[osdcmy] LUN from SAN storage got corrupted

Umarzuki Mochlis
Hi,

Has anyone had problem with QLogic HBA card on Centos 5 that made the
LUN corrupted after awhile?

What I had done (on UCS C200 server with *CentOS 5.5 installed):

- install qlogic driver from its installer CD
- configure multipathing (multipath-tools)
- pvcreate that LUN, create a Logical Volume on its own Volume Group
- configure red hat cluster (rgmanager + cman) with power fencing (APC)
- install zimbra-cluster

What I found from dmesg:

- I/O error on LUNs (standby/ghost path), found from forums that
stated that this is normal (no official documentation to verify this)
- LUN that was formatted to ext3 stated as being mounted for than its
mount threshold and needing file system chek (fsck)
- sometime during boot, centos server stuck during LVM configuration
where it scans for logical volumes

Anyone had encounter this issue before and know any solution/tips?

*at first I tried the latest CentOS 5.8 using default driver provided
by CentOS, but LUNs filesystem got corrpted so I tried CentOS 5.5
which was said to be RHEL 5.5 compatible that statted on QLogic's
driver readme file as tested platform for the provided driver.

--
Regards,

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/
Reply | Threaded
Open this post in threaded view
|

Re: [osdcmy] LUN from SAN storage got corrupted

Sharuzzaman Ahmat Raslan-4
Which machine/system provide the LUN?

It could be corrupted at the provider (storage server), not at the receiver (the server mounting the LUN)



On Thu, Mar 29, 2012 at 12:54 PM, Umarzuki Mochlis <[hidden email]> wrote:
Hi,

Has anyone had problem with QLogic HBA card on Centos 5 that made the
LUN corrupted after awhile?

What I had done (on UCS C200 server with *CentOS 5.5 installed):

- install qlogic driver from its installer CD
- configure multipathing (multipath-tools)
- pvcreate that LUN, create a Logical Volume on its own Volume Group
- configure red hat cluster (rgmanager + cman) with power fencing (APC)
- install zimbra-cluster

What I found from dmesg:

- I/O error on LUNs (standby/ghost path), found from forums that
stated that this is normal (no official documentation to verify this)
- LUN that was formatted to ext3 stated as being mounted for than its
mount threshold and needing file system chek (fsck)
- sometime during boot, centos server stuck during LVM configuration
where it scans for logical volumes

Anyone had encounter this issue before and know any solution/tips?

*at first I tried the latest CentOS 5.8 using default driver provided
by CentOS, but LUNs filesystem got corrpted so I tried CentOS 5.5
which was said to be RHEL 5.5 compatible that statted on QLogic's
driver readme file as tested platform for the provided driver.

--
Regards,

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/



--
Sharuzzaman Ahmat Raslan

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information
 
OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/
 
Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/
Reply | Threaded
Open this post in threaded view
|

Re: [osdcmy] LUN from SAN storage got corrupted

Umarzuki Mochlis
Pada 29 Mac 2012 2:25 PTG, Sharuzzaman Ahmat Raslan
<[hidden email]> menulis:
> Which machine/system provide the LUN?
>
> It could be corrupted at the provider (storage server), not at the receiver
> (the server mounting the LUN)
>

Thanks for replying.

It was a IBM DS 3500 SAN storage but from DS Storage Manager, no error
on hdd or anything.

I tried asking QLogic's suport about this whether there's any special
setting/configuration had to be done on multipath.conf or lvm.conf or
somewhere else but they just point me to RHEL 5 multipath
documentation that never specified anything about QLogic AFAIK.

--
Regards,

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/
Reply | Threaded
Open this post in threaded view
|

Re: [osdcmy] LUN from SAN storage got corrupted

Adzmely Mansor
u r using RHCS, meaning that are multiple servers/centos? and the problematic LUN is a shared fs LUN? or just an active-passive?


On Thu, Mar 29, 2012 at 2:56 PM, Umarzuki Mochlis <[hidden email]> wrote:
Pada 29 Mac 2012 2:25 PTG, Sharuzzaman Ahmat Raslan
<[hidden email]> menulis:
> Which machine/system provide the LUN?
>
> It could be corrupted at the provider (storage server), not at the receiver
> (the server mounting the LUN)
>

Thanks for replying.

It was a IBM DS 3500 SAN storage but from DS Storage Manager, no error
on hdd or anything.

I tried asking QLogic's suport about this whether there's any special
setting/configuration had to be done on multipath.conf or lvm.conf or
somewhere else but they just point me to RHEL 5 multipath
documentation that never specified anything about QLogic AFAIK.

--
Regards,

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information
 
OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/
 
Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/
Reply | Threaded
Open this post in threaded view
|

Re: [osdcmy] LUN from SAN storage got corrupted

Umarzuki Mochlis
Pada 30 Mac 2012 1:23 PG, Adzmely Mansor <[hidden email]> menulis:
> u r using RHCS, meaning that are multiple servers/centos? and the
> problematic LUN is a shared fs LUN? or just an active-passive?
>

2+1 setup where the LUN will be mounted on standy server upon being fenced

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/
Reply | Threaded
Open this post in threaded view
|

Re: [osdcmy] LUN from SAN storage got corrupted

E A Faisal
Are active & passive machines' time synced to NTP server?

On Fri, Mar 30, 2012 at 8:10 AM, Umarzuki Mochlis <[hidden email]> wrote:
Pada 30 Mac 2012 1:23 PG, Adzmely Mansor <[hidden email]> menulis:
> u r using RHCS, meaning that are multiple servers/centos? and the
> problematic LUN is a shared fs LUN? or just an active-passive?
>

2+1 setup where the LUN will be mounted on standy server upon being fenced

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information
 
OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/
 
Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/
Reply | Threaded
Open this post in threaded view
|

Re: [osdcmy] LUN from SAN storage got corrupted

Umarzuki Mochlis
yes

later found out it was kernel bug on centos 5 regarding to lvm2
i don't remember what fix/workaround that my colleague done but
there's no more issue with suddenly corrupted file system

Pada 30 Mac 2012 11:11 PG, E A Faisal <[hidden email]> menulis:

> Are active & passive machines' time synced to NTP server?
>
> On Fri, Mar 30, 2012 at 8:10 AM, Umarzuki Mochlis <[hidden email]>
> wrote:
>>
>> Pada 30 Mac 2012 1:23 PG, Adzmely Mansor <[hidden email]> menulis:
>> > u r using RHCS, meaning that are multiple servers/centos? and the
>> > problematic LUN is a shared fs LUN? or just an active-passive?
>> >
>>
>> 2+1 setup where the LUN will be mounted on standy server upon being fenced
>>

-
Regards,

Umarzuki Mochlis
http://debmal.my

--
To unsubscribe from and detail about this group http://portal.mosc.my/osdc-my-mailing-list-information

OSDC.my Discussion Group In Facebook
http://www.facebook.com/groups/osdcmalaysia/

Malaysia Open Source Conference 2012
MOSC2012 http://portal.mosc.my/