Filesystem Size Used Avail Use% Mounted on /dev/md2 1016G 212G 754G 22% / tmpfs 7.9G 0 7.9G 0% /lib/init/rw proc 0 0 0 - /proc sysfs 0 0 0 - /sys udev 7.9G 164K 7.9G 1% /dev tmpfs 7.9G 0 7.9G 0% /dev/shm devpts 0 0 0 - /dev/pts /dev/md1 496M 38M 433M 8% /boot /dev/md3 1.7T 712G 926G 44% /backups ================================== Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md3 : active raid1 sdb4[2] sda4[0] 1822442815 blocks super 1.2 [2/2] [UU] md2 : active raid1 sdb3[2] sda3[0] 1073740664 blocks super 1.2 [2/2] [UU] md1 : active raid1 sdb2[2] sda2[0] 524276 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdb1[2] sda1[0] 33553336 blocks super 1.2 [2/2] [UU] unused devices: ================================== smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF READ SMART DATA SECTION === SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 115 096 006 Pre-fail Always - 99990792 3 Spin_Up_Time 0x0003 094 094 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 7 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 083 060 030 Pre-fail Always - 208878781 9 Power_On_Hours 0x0032 082 082 000 Old_age Always - 16557 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 7 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 172 188 Command_Timeout 0x0032 100 099 000 Old_age Always - 21475164170 189 High_Fly_Writes 0x003a 094 094 000 Old_age Always - 6 190 Airflow_Temperature_Cel 0x0022 063 058 045 Old_age Always - 37 (Lifetime Min/Max 31/39) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 3 193 Load_Cycle_Count 0x0032 099 099 000 Old_age Always - 2290 194 Temperature_Celsius 0x0022 037 042 000 Old_age Always - 37 (0 19 0 0) 197 Current_Pending_Sector 0x0012 100 001 000 Old_age Always - 32 198 Offline_Uncorrectable 0x0010 100 001 000 Old_age Offline - 32 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 11 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 247811023060986 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 106141094422511 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 73353559836298 smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF READ SMART DATA SECTION === SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 102 095 006 Pre-fail Always - 4872544 3 Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 31 5 Reallocated_Sector_Ct 0x0033 099 099 036 Pre-fail Always - 2128 7 Seek_Error_Rate 0x000f 062 051 030 Pre-fail Always - 2169788739005 9 Power_On_Hours 0x0032 085 085 000 Old_age Always - 13183 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 31 183 Runtime_Bad_Block 0x0032 099 099 000 Old_age Always - 1 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 255 188 Command_Timeout 0x0032 100 099 000 Old_age Always - 8590065667 189 High_Fly_Writes 0x003a 095 095 000 Old_age Always - 5 190 Airflow_Temperature_Cel 0x0022 061 049 045 Old_age Always - 39 (Lifetime Min/Max 29/41) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 25 193 Load_Cycle_Count 0x0032 093 093 000 Old_age Always - 14868 194 Temperature_Celsius 0x0022 039 051 000 Old_age Always - 39 (0 22 0 0) 197 Current_Pending_Sector 0x0012 099 099 000 Old_age Always - 208 198 Offline_Uncorrectable 0x0010 099 099 000 Old_age Offline - 208 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 190541929132815 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 9837679545946 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 39117646563283 smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net Smartctl open device: /dev/sdc failed: No such device smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net Smartctl open device: /dev/sdd failed: No such device ================================== Mar 31 04:14:33 s25 kernel: [7179988.165498] res 41/40:20:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:33 s25 kernel: [7179988.165610] ata2.00: error: { UNC } Mar 31 04:14:36 s25 kernel: [7179991.040939] res 41/40:20:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:36 s25 kernel: [7179991.041050] ata2.00: error: { UNC } Mar 31 04:14:39 s25 kernel: [7179994.039335] res 41/40:20:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:39 s25 kernel: [7179994.039446] ata2.00: error: { UNC } Mar 31 04:14:41 s25 kernel: [7179996.937873] res 41/40:20:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:41 s25 kernel: [7179996.937985] ata2.00: error: { UNC } Mar 31 04:14:44 s25 kernel: [7179999.828154] res 41/40:20:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:44 s25 kernel: [7179999.828266] ata2.00: error: { UNC } Mar 31 04:14:47 s25 kernel: [7180002.718437] res 41/40:20:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:47 s25 kernel: [7180002.718548] ata2.00: error: { UNC } Mar 31 04:14:47 s25 kernel: [7180002.738374] sd 1:0:0:0: [sdb] Unhandled sense code Mar 31 04:14:47 s25 kernel: [7180002.738400] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Mar 31 04:14:47 s25 kernel: [7180002.738445] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Mar 31 04:14:47 s25 kernel: [7180002.738590] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Mar 31 04:14:47 s25 kernel: [7180002.738639] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc 80 00 00 20 00 Mar 31 04:14:47 s25 kernel: [7180002.738681] end_request: I/O error, dev sdb, sector 1963449496 Mar 31 04:14:47 s25 kernel: [7180002.738710] raid1: sdb3: rescheduling sector 1895285888 Mar 31 04:14:50 s25 kernel: [7180005.712094] res 41/40:08:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:50 s25 kernel: [7180005.712206] ata2.00: error: { UNC } Mar 31 04:14:53 s25 kernel: [7180008.573969] res 41/40:08:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:53 s25 kernel: [7180008.574080] ata2.00: error: { UNC } Mar 31 04:14:56 s25 kernel: [7180011.430956] res 41/40:08:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:56 s25 kernel: [7180011.431067] ata2.00: error: { UNC } Mar 31 04:14:59 s25 kernel: [7180014.279621] res 41/40:08:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:14:59 s25 kernel: [7180014.279732] ata2.00: error: { UNC } Mar 31 04:15:02 s25 kernel: [7180017.128199] res 41/40:08:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:15:02 s25 kernel: [7180017.128311] ata2.00: error: { UNC } Mar 31 04:15:05 s25 kernel: [7180019.976832] res 41/40:08:98:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Mar 31 04:15:05 s25 kernel: [7180019.976943] ata2.00: error: { UNC } Mar 31 04:15:05 s25 kernel: [7180020.005139] sd 1:0:0:0: [sdb] Unhandled sense code Mar 31 04:15:05 s25 kernel: [7180020.005165] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Mar 31 04:15:05 s25 kernel: [7180020.005210] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Mar 31 04:15:05 s25 kernel: [7180020.005355] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Mar 31 04:15:05 s25 kernel: [7180020.005404] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc 98 00 00 08 00 Mar 31 04:15:05 s25 kernel: [7180020.005446] end_request: I/O error, dev sdb, sector 1963449496 Mar 31 04:15:05 s25 kernel: [7180020.023404] raid1:md2: read error corrected (8 sectors at 1895287960 on sdb3) Mar 31 04:15:05 s25 kernel: [7180020.023459] raid1: sdb3: redirecting sector 1895285888 to another mirror Apr 1 10:10:43 s25 kernel: [7287703.717046] res 41/40:20:90:09:f4/00:00:73:00:00/00 Emask 0x409 (media error) Apr 1 10:10:43 s25 kernel: [7287703.717158] ata2.00: error: { UNC } Apr 1 10:10:46 s25 kernel: [7287706.615635] res 41/40:20:90:09:f4/00:00:73:00:00/00 Emask 0x409 (media error) Apr 1 10:10:46 s25 kernel: [7287706.615747] ata2.00: error: { UNC } Apr 2 06:00:26 s25 kernel: [7359050.190540] res 41/40:20:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:26 s25 kernel: [7359050.190651] ata2.00: error: { UNC } Apr 2 06:00:29 s25 kernel: [7359053.030917] res 41/40:20:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:29 s25 kernel: [7359053.031029] ata2.00: error: { UNC } Apr 2 06:00:32 s25 kernel: [7359055.862820] res 41/40:20:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:32 s25 kernel: [7359055.862931] ata2.00: error: { UNC } Apr 2 06:00:34 s25 kernel: [7359058.694798] res 41/40:20:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:34 s25 kernel: [7359058.694909] ata2.00: error: { UNC } Apr 2 06:00:37 s25 kernel: [7359061.526774] res 41/40:20:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:37 s25 kernel: [7359061.526885] ata2.00: error: { UNC } Apr 2 06:00:40 s25 kernel: [7359064.264971] res 41/40:20:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:40 s25 kernel: [7359064.265082] ata2.00: error: { UNC } Apr 2 06:00:40 s25 kernel: [7359064.378327] sd 1:0:0:0: [sdb] Unhandled sense code Apr 2 06:00:40 s25 kernel: [7359064.378353] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 2 06:00:40 s25 kernel: [7359064.378398] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Apr 2 06:00:40 s25 kernel: [7359064.378545] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Apr 2 06:00:40 s25 kernel: [7359064.378594] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc a0 00 00 20 00 Apr 2 06:00:40 s25 kernel: [7359064.378636] end_request: I/O error, dev sdb, sector 1963449504 Apr 2 06:00:40 s25 kernel: [7359064.378665] raid1: sdb3: rescheduling sector 1895285920 Apr 2 06:00:43 s25 kernel: [7359067.361528] res 41/40:08:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:43 s25 kernel: [7359067.361639] ata2.00: error: { UNC } Apr 2 06:00:46 s25 kernel: [7359070.231485] res 41/40:08:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:46 s25 kernel: [7359070.231596] ata2.00: error: { UNC } Apr 2 06:00:49 s25 kernel: [7359073.165421] res 41/40:08:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:49 s25 kernel: [7359073.165532] ata2.00: error: { UNC } Apr 2 06:00:52 s25 kernel: [7359076.107164] res 41/40:08:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:52 s25 kernel: [7359076.107275] ata2.00: error: { UNC } Apr 2 06:00:55 s25 kernel: [7359079.048930] res 41/40:08:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:55 s25 kernel: [7359079.049040] ata2.00: error: { UNC } Apr 2 06:00:58 s25 kernel: [7359081.998978] res 41/40:08:a0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:00:58 s25 kernel: [7359082.000585] ata2.00: error: { UNC } Apr 2 06:00:58 s25 kernel: [7359082.071372] sd 1:0:0:0: [sdb] Unhandled sense code Apr 2 06:00:58 s25 kernel: [7359082.071397] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 2 06:00:58 s25 kernel: [7359082.071442] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Apr 2 06:00:58 s25 kernel: [7359082.071588] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Apr 2 06:00:58 s25 kernel: [7359082.071637] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc a0 00 00 08 00 Apr 2 06:00:58 s25 kernel: [7359082.071679] end_request: I/O error, dev sdb, sector 1963449504 Apr 2 06:00:58 s25 kernel: [7359082.106676] raid1:md2: read error corrected (8 sectors at 1895287968 on sdb3) Apr 2 06:01:01 s25 kernel: [7359085.380109] res 41/40:08:a8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:01 s25 kernel: [7359085.380221] ata2.00: error: { UNC } Apr 2 06:01:04 s25 kernel: [7359088.245862] res 41/40:08:a8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:04 s25 kernel: [7359088.245973] ata2.00: error: { UNC } Apr 2 06:01:07 s25 kernel: [7359091.145967] res 41/40:08:a8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:07 s25 kernel: [7359091.146094] ata2.00: error: { UNC } Apr 2 06:01:10 s25 kernel: [7359094.171001] res 41/40:08:a8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:10 s25 kernel: [7359094.171112] ata2.00: error: { UNC } Apr 2 06:01:13 s25 kernel: [7359097.171074] res 41/40:08:a8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:13 s25 kernel: [7359097.171186] ata2.00: error: { UNC } Apr 2 06:01:16 s25 kernel: [7359100.129534] res 41/40:08:a8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:16 s25 kernel: [7359100.129646] ata2.00: error: { UNC } Apr 2 06:01:16 s25 kernel: [7359100.202089] sd 1:0:0:0: [sdb] Unhandled sense code Apr 2 06:01:16 s25 kernel: [7359100.202115] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 2 06:01:16 s25 kernel: [7359100.202160] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Apr 2 06:01:16 s25 kernel: [7359100.202307] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Apr 2 06:01:16 s25 kernel: [7359100.202356] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc a8 00 00 08 00 Apr 2 06:01:16 s25 kernel: [7359100.202400] end_request: I/O error, dev sdb, sector 1963449512 Apr 2 06:01:16 s25 kernel: [7359100.207184] raid1:md2: read error corrected (8 sectors at 1895287976 on sdb3) Apr 2 06:01:19 s25 kernel: [7359103.455874] res 41/40:08:b0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:19 s25 kernel: [7359103.455986] ata2.00: error: { UNC } Apr 2 06:01:22 s25 kernel: [7359106.379488] res 41/40:08:b0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:22 s25 kernel: [7359106.379599] ata2.00: error: { UNC } Apr 2 06:01:25 s25 kernel: [7359109.313445] res 41/40:08:b0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:25 s25 kernel: [7359109.313557] ata2.00: error: { UNC } Apr 2 06:01:28 s25 kernel: [7359112.255187] res 41/40:08:b0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:28 s25 kernel: [7359112.255311] ata2.00: error: { UNC } Apr 2 06:01:31 s25 kernel: [7359115.196954] res 41/40:08:b0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:31 s25 kernel: [7359115.197066] ata2.00: error: { UNC } Apr 2 06:01:34 s25 kernel: [7359118.130372] res 41/40:08:b0:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:34 s25 kernel: [7359118.130483] ata2.00: error: { UNC } Apr 2 06:01:34 s25 kernel: [7359118.194059] sd 1:0:0:0: [sdb] Unhandled sense code Apr 2 06:01:34 s25 kernel: [7359118.194084] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 2 06:01:34 s25 kernel: [7359118.194130] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Apr 2 06:01:34 s25 kernel: [7359118.194276] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Apr 2 06:01:34 s25 kernel: [7359118.194325] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc b0 00 00 08 00 Apr 2 06:01:34 s25 kernel: [7359118.194367] end_request: I/O error, dev sdb, sector 1963449520 Apr 2 06:01:34 s25 kernel: [7359118.199545] raid1:md2: read error corrected (8 sectors at 1895287984 on sdb3) Apr 2 06:01:37 s25 kernel: [7359121.467002] res 41/40:08:b8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:37 s25 kernel: [7359121.467118] ata2.00: error: { UNC } Apr 2 06:01:40 s25 kernel: [7359124.375717] res 41/40:08:b8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:40 s25 kernel: [7359124.375828] ata2.00: error: { UNC } Apr 2 06:01:43 s25 kernel: [7359127.325811] res 41/40:08:b8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:43 s25 kernel: [7359127.325923] ata2.00: error: { UNC } Apr 2 06:01:46 s25 kernel: [7359130.317548] res 41/40:08:b8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:46 s25 kernel: [7359130.317660] ata2.00: error: { UNC } Apr 2 06:01:49 s25 kernel: [7359133.300932] res 41/40:08:b8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:49 s25 kernel: [7359133.301043] ata2.00: error: { UNC } Apr 2 06:01:52 s25 kernel: [7359136.309311] res 41/40:08:b8:dc:07/00:00:75:00:00/00 Emask 0x409 (media error) Apr 2 06:01:52 s25 kernel: [7359136.309422] ata2.00: error: { UNC } Apr 2 06:01:53 s25 kernel: [7359136.749056] sd 1:0:0:0: [sdb] Unhandled sense code Apr 2 06:01:53 s25 kernel: [7359136.749081] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 2 06:01:53 s25 kernel: [7359136.749126] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Apr 2 06:01:53 s25 kernel: [7359136.749272] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Apr 2 06:01:53 s25 kernel: [7359136.749321] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 75 07 dc b8 00 00 08 00 Apr 2 06:01:53 s25 kernel: [7359136.749363] end_request: I/O error, dev sdb, sector 1963449528 Apr 2 06:01:53 s25 kernel: [7359136.971882] raid1:md2: read error corrected (8 sectors at 1895287992 on sdb3) Apr 2 06:01:53 s25 kernel: [7359136.971939] raid1: sdb3: redirecting sector 1895285920 to another mirror Apr 4 01:02:46 s25 kernel: [7513911.833147] res 41/40:a0:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:02:46 s25 kernel: [7513911.833230] ata2.00: error: { UNC } Apr 4 01:02:49 s25 kernel: [7513914.868194] res 41/40:a0:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:02:49 s25 kernel: [7513914.868306] ata2.00: error: { UNC } Apr 4 01:02:53 s25 kernel: [7513917.892217] res 41/40:a0:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:02:53 s25 kernel: [7513917.892329] ata2.00: error: { UNC } Apr 4 01:02:55 s25 kernel: [7513920.878918] res 41/40:a0:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:02:55 s25 kernel: [7513920.879030] ata2.00: error: { UNC } Apr 4 01:02:58 s25 kernel: [7513923.774364] res 41/40:a0:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:02:58 s25 kernel: [7513923.774476] ata2.00: error: { UNC } Apr 4 01:03:06 s25 kernel: [7513931.611260] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.611402] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.611543] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.613180] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.613320] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.613461] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.613601] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.613741] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.613882] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614022] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614163] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614304] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614444] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614585] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614726] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.614866] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615007] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615165] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615315] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615468] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615620] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615774] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.615927] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616080] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616234] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616387] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616539] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616693] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616846] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.616998] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:06 s25 kernel: [7513931.617150] res 40/00:d0:00:eb:30/00:00:18:00:00/40 Emask 0x1 (device error) Apr 4 01:03:07 s25 kernel: [7513932.056081] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 4 01:03:07 s25 kernel: [7513932.056127] sd 1:0:0:0: [sdb] Sense Key : Aborted Command [current] [descriptor] Apr 4 01:03:07 s25 kernel: [7513932.056274] sd 1:0:0:0: [sdb] Add. Sense: No additional sense information Apr 4 01:03:07 s25 kernel: [7513932.057803] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 69 d0 26 00 00 00 a0 00 Apr 4 01:03:07 s25 kernel: [7513932.057846] end_request: I/O error, dev sdb, sector 1775248896 Apr 4 01:03:07 s25 kernel: [7513932.057875] raid1: sdb3: rescheduling sector 1707085312 Apr 4 01:03:07 s25 kernel: [7513932.057904] raid1: sdb3: rescheduling sector 1707085320 Apr 4 01:03:07 s25 kernel: [7513932.057931] raid1: sdb3: rescheduling sector 1707085328 Apr 4 01:03:07 s25 kernel: [7513932.057959] raid1: sdb3: rescheduling sector 1707085336 Apr 4 01:03:07 s25 kernel: [7513932.057986] raid1: sdb3: rescheduling sector 1707085344 Apr 4 01:03:07 s25 kernel: [7513932.058014] raid1: sdb3: rescheduling sector 1707085352 Apr 4 01:03:07 s25 kernel: [7513932.058042] raid1: sdb3: rescheduling sector 1707085360 Apr 4 01:03:07 s25 kernel: [7513932.058069] raid1: sdb3: rescheduling sector 1707085368 Apr 4 01:03:07 s25 kernel: [7513932.058097] raid1: sdb3: rescheduling sector 1707085376 Apr 4 01:03:07 s25 kernel: [7513932.058123] raid1: sdb3: rescheduling sector 1707085384 Apr 4 01:03:10 s25 kernel: [7513935.249032] res 41/40:08:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:03:10 s25 kernel: [7513935.249144] ata2.00: error: { UNC } Apr 4 01:03:13 s25 kernel: [7513938.210334] res 41/40:08:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:03:13 s25 kernel: [7513938.210446] ata2.00: error: { UNC } Apr 4 01:03:16 s25 kernel: [7513941.058963] res 41/40:08:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:03:16 s25 kernel: [7513941.059074] ata2.00: error: { UNC } Apr 4 01:03:19 s25 kernel: [7513943.907678] res 41/40:08:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:03:19 s25 kernel: [7513943.907789] ata2.00: error: { UNC } Apr 4 01:03:21 s25 kernel: [7513946.756229] res 41/40:08:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:03:21 s25 kernel: [7513946.756341] ata2.00: error: { UNC } Apr 4 01:03:24 s25 kernel: [7513949.604842] res 41/40:08:58:26:d0/00:00:69:00:00/00 Emask 0x409 (media error) Apr 4 01:03:24 s25 kernel: [7513949.604953] ata2.00: error: { UNC } Apr 4 01:03:24 s25 kernel: [7513949.624880] sd 1:0:0:0: [sdb] Unhandled sense code Apr 4 01:03:24 s25 kernel: [7513949.624905] sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Apr 4 01:03:24 s25 kernel: [7513949.624950] sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor] Apr 4 01:03:24 s25 kernel: [7513949.625096] sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed Apr 4 01:03:24 s25 kernel: [7513949.625146] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 69 d0 26 58 00 00 08 00 Apr 4 01:03:24 s25 kernel: [7513949.625188] end_request: I/O error, dev sdb, sector 1775248984 Apr 4 01:03:24 s25 kernel: [7513949.691871] raid1:md2: read error corrected (8 sectors at 1707087448 on sdb3) Apr 4 01:03:24 s25 kernel: [7513949.691963] raid1: sda3: redirecting sector 1707085400 to another mirror Apr 4 01:03:25 s25 kernel: [7513949.897850] raid1: sda3: redirecting sector 1707085408 to another mirror Apr 4 01:03:25 s25 kernel: [7513949.951418] raid1: sda3: redirecting sector 1707085416 to another mirror Apr 4 01:03:25 s25 kernel: [7513949.969023] raid1: sda3: redirecting sector 1707085424 to another mirror Apr 4 01:03:25 s25 kernel: [7513949.985887] raid1: sda3: redirecting sector 1707085432 to another mirror Apr 4 01:03:25 s25 kernel: [7513950.000708] raid1: sda3: redirecting sector 1707085440 to another mirror Apr 4 01:03:25 s25 kernel: [7513950.006566] raid1: sda3: redirecting sector 1707085448 to another mirror Apr 4 01:03:25 s25 kernel: [7513950.007062] raid1: sda3: redirecting sector 1707085456 to another mirror Apr 4 01:03:25 s25 kernel: [7513950.016626] raid1: sda3: redirecting sector 1707085464 to another mirror Apr 4 01:47:44 s25 kernel: [7516608.080308] res 41/40:00:40:f3:97/00:02:2e:00:00/00 Emask 0x409 (media error) Apr 4 01:47:44 s25 kernel: [7516608.080419] ata2.00: error: { UNC } Apr 4 01:47:48 s25 kernel: [7516612.155036] res 41/40:00:48:f3:97/00:02:2e:00:00/00 Emask 0x409 (media error) Apr 4 01:47:48 s25 kernel: [7516612.155147] ata2.00: error: { UNC } Apr 4 01:47:51 s25 kernel: [7516615.476978] res 41/40:00:48:f3:97/00:02:2e:00:00/00 Emask 0x409 (media error) Apr 4 01:47:51 s25 kernel: [7516615.477088] ata2.00: error: { UNC } Apr 4 03:08:24 s25 kernel: [7521445.559554] res 41/40:08:f0:c1:10/00:00:73:00:00/00 Emask 0x409 (media error) Apr 4 03:08:24 s25 kernel: [7521445.559665] ata2.00: error: { UNC } Apr 4 03:08:34 s25 kernel: [7521455.096221] res 41/40:08:f8:c1:10/00:00:73:00:00/00 Emask 0x409 (media error) Apr 4 03:08:34 s25 kernel: [7521455.096333] ata2.00: error: { UNC } Apr 4 03:08:37 s25 kernel: [7521458.163997] res 41/40:08:f8:c1:10/00:00:73:00:00/00 Emask 0x409 (media error) Apr 4 03:08:37 s25 kernel: [7521458.165607] ata2.00: error: { UNC } Apr 4 03:08:39 s25 kernel: [7521461.032626] res 41/40:08:f8:c1:10/00:00:73:00:00/00 Emask 0x409 (media error) Apr 4 03:08:39 s25 kernel: [7521461.032737] ata2.00: error: { UNC }