Announcement: A GOOD Virus

For discussions about security.
Message
Author
Bindee

#21 Post by Bindee »

Sorry to hear it's being awkward. :(

They recently claimed that DDR4 was safe from this but there have now been a couple of reports of DDR4 giving errors on the passmark memtest forum.

Sigh!

:!:

Bindee

#22 Post by Bindee »

I tested 16gb last night and almost gave up the will to live...........

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#23 Post by otropogo »

This week I received replacement HyperX RAM from Kingston and tested it. The Row Hammer test produced several times more errors in this brand new RAM than it did in the (same spec) RAM I've used for almost 2 years (24 to 55 errors per pass vs. 3 to 6 errors).

I was pretty upset and complained heatedly to Kingston, and finally got to speak to a senior technical support person who informed me that:

1. the new RAM was thoroughly tested on the same chipset as my laptop's using the same Passmark Row Hammer test as I had, and passed.

2. RAM does not become more vulnerable over time from wear and tear (which seems to be borne out by my own testing).The explanation offered for the higher error rate on the newer RAM was greater miniaturization and closer spacing on the current SODIMMS.

3. ECC RAM is not invulnerable to the Row Hammer exploit, as has been suggested

4. vulnerability to the Row Hammer Exploit can be mitigated via the BIOS, and the RAM that failed on my system might pass if I had a current BIOS installed.

5. the HyperX RAM in question, both the old and the new, is overclocked, and this makes it more likely to produce errors on the Row Hammer test

(This seems to be supported by my tests on the stock RAM that Asus bundled with the system, which is also Kingston RAM, but set to run only at industry standard speed - with timing of 11-11-11-28, as opposed to 9-9-9-27), and which produces no errors in seven consecutive passes of the Row Hammer Test.

I pass this on for what it's worth. I don't have the technical knowledge to assess these statements. But I've accepted Kingston's offer to send me an industry standard 16GB kit, with the same specs as my original 8GB kit, to try.

If that also fails, the only remaining suggestion is a BIOS upgrade, which I would only venture if I were sure that it could be reversed. I'm not at all happy with the AMI BIOS that shipped with the laptop. It's easily the most crippled and least user friendly BIOS I can remember.

But when a company sells a laptop that can exhaust the battery in only two hours and then threatens to void the warranty if anyone except their service centre at the other end of the continent accesses that battery, as ASUS did, then I can easily imagine them offering an even worse BIOS as an upgrade.
otropogo@gmail.com facebook.com/otropogo

Bindee

#24 Post by Bindee »

Cheers for the update and sorry to hear that it's still a problem. :(

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#25 Post by otropogo »

Bindee wrote:Cheers for the update and sorry to hear that it's still a problem. :(
You're welcome. And to put the problem into perspective, I was given to understand that Kingston has known about the issue since 2012, but considered it essentially a server problem, not a likely issue for a PC user.

I was also told that in order to use the row hammer exploit to intrude on or take control of a PC, the attacker would have to have physical access to the PC. And if that's an accurate assessment, then perhaps the majority of home PC users have no cause for concern.

OTOH, anyone who seriously upsets government, big business, or just garden variety criminals, has plenty of reason to be worried. Few people have the means to keep anything as large as a laptop physically secure 24/7.

And once in, hiding a few gigabytes of illicit material where the owner of the machine will never discover it is probably the work of a few minutes, if that. For police to find it, with the help of an anonymous tip, is a snap. Game over.

Your life as a member of society can be over before your first court appearance, and the trial, if you're stubborn enough to plead not guilty, is a formality that can do nothing to give it back to you.
otropogo@gmail.com facebook.com/otropogo

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#26 Post by otropogo »

A few days ago I received a second 16GB replacement kit for my HyperX RAM from Kingston.

This kit runs at "industry standard" settings of 11-11-11-28, has a longer latency period, and is rated at a somewhat slower transfer rate.

My informant at Kingston also mentioned that it has some Row Hammer "mitigation", but didn't elaborate.

I ran 8 passes of the whole free memtest 6.01.0 battery , as well as a couple of extra passes of Row Hammer alone, and no errors were reported.

The one disturbing result is that by the end of the 8 pass series, the tests were taking almost twice as long to run as at the beginning, and the the Row Hammer test, in particular, slowed down massively in the eighth pass, taking 57 minutes to complete, whereas in pass 7 it took only 42. The first Row Hammer pass only required 30 minutes. The entire battery completed in only 57 minutes on the first pass, but took 110 minutes on the last, so it wasn't just Row Hammer that slowed down.

I took screen shots of the first pass and of the last two (slept in between - the series took 14 hours), and noted that while cpu temperature went up with certain tests, it went back down during Row Hammer. And the overall temperature didn't increase appreciably (the early low was 71C, the ending low was 73C) The highest recorded was 80C.

The laptop was sitting on a metal cooling stand with two externally powered fans, and the ambient temperature of 25C didn't change by more than half a degree during the test. It was also offline and couldn't write to the boot medium, which was write protected.

I'm hoping to get an explanation. At the end I was worried enough to decide against running a final pass of just Row Hammer.

The image below is of the RAM that passed.
Attachments
IMG_4660c.jpg
(11.67 KiB) Downloaded 413 times
otropogo@gmail.com facebook.com/otropogo

Bindee

#27 Post by Bindee »

I had the same with 16gb desktop of desktop ram , The first pass was fast but every pass after that got slower and slower until it was at a crawl.

But that's to be expected as you are electively DDOS'ing the ram and forcing continuous flipping of data storage and rewriting of the cells to try and force the capacitors to leak.

Hence why it's called hammering :P

Passmark recommend 4 passes of test #13 alone to be sure as it's not until it's at a crawl that some sticks start to leak , infact i'm not sure why they include it with the other test patterns instead of having it as a separate 4 run test on its own at the end of all the others as when you restart from #1 you are clearing the ram so you don't get the same hammering or slow down.

Temps wise it was not as extreme as your laptop but it was higher than normal , it started around the mid to high 30's and was in the mid to high 50's by the end of the test with the fan spinning up faster.

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#28 Post by otropogo »

Bindee wrote:.... when you restart from #1 you are clearing the ram so you don't get the same hammering or slow down.

Temps wise it was not as extreme as your laptop but it was higher than normal , it started around the mid to high 30's and was in the mid to high 50's by the end of the test with the fan spinning up faster.
How much did your system slow down?

My 8 passes of the complete series of test were run back to back. Yet the last Pass took twice as long as the first, and not just the Row Hammer. If clearing the RAM prevents that, it didn't happen here.

The Row Hammer test was also the coolest running test, at 73C, while tests 5,6,8,9 and 10 all ran at 78 to 80C. And there was no increase in the temperature range over the 14 hours of testing. The first test started at 71C, and it was 72C when the last pass was done.

Still waiting for Kingston's explanation.
Attachments
IMG_4703d.jpg
(53.81 KiB) Downloaded 239 times
IMG_4712d.jpg
(54.46 KiB) Downloaded 226 times
otropogo@gmail.com facebook.com/otropogo

Bindee

#29 Post by Bindee »

It was a few weeks ago that i done 4 passes of #13 but from what i remember it was like 3 hours , Pass 1 was just over 20mins and pass 4 was something like an hour and 10 mins.

The difference was quiet extreme.

I was testing the ram for a friend and i removed it straight after the test and it was also very hot to touch so the test really thrashes the hell out of the ram.

I can see why people complain that the hammering or changing the refresh rate of the ram can ruin battery life of laptops , i wrongly had the impression that ram hardly used any power.

Scooby
Posts: 599
Joined: Sat 03 Mar 2012, 09:04

#30 Post by Scooby »

Might be able to detect a rowhammer attack

[quote]
Modern CPUs provide mechanisms that allow monitoring of cache misses for purposes of performance
analysis. These mechanisms can be repurposed by a defender to monitor the system for sudden bursts of
cache misses, as truly cache­pessimal access patterns appear to be rare in typical laptop and desktop
workloads. By measuring “time elapsed per N cache misses

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#31 Post by otropogo »

Bindee wrote:It was a few weeks ago that i done 4 passes of #13 but from what i remember it was like 3 hours , Pass 1 was just over 20mins and pass 4 was something like an hour and 10 mins.

The difference was quiet extreme.
If you ran the memtest battery, except for the Row Hammer test, you'd find similar slowing. At least that's what my tests showed.
Bindee wrote:I was testing the ram for a friend and i removed it straight after the test and it was also very hot to touch so the test really thrashes the hell out of the ram.
70-80C is pretty hot. And that's what my cpu temp was reported to be at. Most residential water heaters have thermostats keeping the water below 65C, and that's scalding. I would think that if the cpu and its caches can handle 70-80C, SODIMMs you can hold in your fingers should be ok.


Bindee wrote:I can see why people complain that the hammering or changing the refresh rate of the ram can ruin battery life of laptops , i wrongly had the impression that ram hardly used any power.
Well, I had no way to measure the temperature of the SODIMMs, and I didn't remove them after the tests. But it seems to me that their temperature would go up with the temperature of the cpu, and that temperature dropped from 80C to 73C every time the Rowhammer test started, and it stayed at 73 for the entire test, even when it took 57 minutes.

So if temperature is what damages the battery and the RAM, then the other tests in the Memtest series would seem to be more damaging.

BTW - my informant at Kingston assured me that the SODIMM kit he'd sent me last (described and picture above) was tested for 66 consecutive hours with the Row Hammer test before being shipped out to me.
otropogo@gmail.com facebook.com/otropogo

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#32 Post by otropogo »

[quote="Scooby"]Might be able to detect a rowhammer attack

[quote]
Modern CPUs provide mechanisms that allow monitoring of cache misses for purposes of performance
analysis. These mechanisms can be repurposed by a defender to monitor the system for sudden bursts of
cache misses, as truly cache­pessimal access patterns appear to be rare in typical laptop and desktop
workloads. By measuring “time elapsed per N cache misses
otropogo@gmail.com facebook.com/otropogo

Bindee

#33 Post by Bindee »

The heat doesn't do damage , the hammering or changing the refresh just drains the battery quicker because of the constant high power draw.

You reboot and clear the ram and the first pass will be quick again.

Scooby
Posts: 599
Joined: Sat 03 Mar 2012, 09:04

#34 Post by Scooby »

otropogo wrote: Whom are you quoting?
http://googleprojectzero.blogspot.fr/20 ... -gain.html

I actually got the quote from a pdf that I was unable to find however
the text is the same, you can check for your self

otropogo wrote: Kingston tells me that in order to use the Row Hammer exploit to hack a system, you have to have physical access to the PC or server.

Your quotes above seem to suggest attacks via an internet connection are possible.
http://arxiv.org/abs/1507.06955

http://arxiv.org/pdf/1507.06955v2.pdf



"In this paper we present Rowhammer.js, a JavaScript-based
implementation of the Rowhammer attack. Our attack uses an eviction
strategy found by a generic algorithm that improves the eviction rate
compared to existing eviction strategies from 95.2% to 99.99%.
Rowhammer.js is the first remote software-induced hardware-fault attack.
In contrast to other fault attacks it does not require physical access to
the machine
, or the execution of native code or access to special
instructions. As JavaScript-based fault attacks can be performed on
millions of users stealthily and simultaneously, we propose
countermeasures that can be implemented immediately. "

Seems like Kingston is lying to you!?

See also the link in the first post.


Made a quick test of cache misses during 10 seconds.


15115716
373477355


first number, 15115716, is without rowhammering
second running rowhammer-test


the difference in misses are quite drastic

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#35 Post by otropogo »

Bindee wrote:

PostPosted: Sat 05 Sep 2015, 04:20 Post subject:
I had the same with 16gb desktop of desktop ram , The first pass was fast but every pass after that got slower and slower until it was at a crawl.

But that's to be expected as you are electively DDOS'ing the ram and forcing continuous flipping of data storage and rewriting of the cells to try and force the capacitors to leak.

Hence why it's called hammering Razz....
Explanation of strange quote: I logged on to the forum, and wanted to quote the older post on Page 2 of the thread, but when I moved from page 1 to page 2 or from page 3 to page too, I was no longer "logged in", and so couldn't quote! I had to copy , then move to page three and paste.


Bindee wrote:Passmark recommend 4 passes of test #13 alone to be sure as it's not until it's at a crawl that some sticks start to leak , infact i'm not sure why they include it with the other test patterns instead of having it as a separate 4 run test on its own at the end of all the others as when you restart from #1 you are clearing the ram so you don't get the same hammering or slow down.

Temps wise it was not as extreme as your laptop but it was higher than normal , it started around the mid to high 30's and was in the mid to high 50's by the end of the test with the fan spinning up faster.



The heat doesn't do damage , the hammering or changing the refresh just drains the battery quicker because of the constant high power draw.

You reboot and clear the ram and the first pass will be quick again.
As I mentioned in my previous post, your conclusion that when running several passes of the whole memtest battery, RAM is cleared, and Row Hammer runs at full speed as at the start on the next pass.

In order to get a better perspective on this (since I'm not going to sit watching memtest run steadily for 14 hours with camera in hand to document the changing temps and speeds), I asked Kingston for the report of the 66 hour Row Hammer only series I was told was run on my last 16GB kit (the one that passed when I ran it for 14 hours), and it was kindly provided.

Sadly, it was a strictly text log (probably necessary because of the amount of disk space a regular report would have consumed - the log was some 300KB as it was). It also had no cpu temperature measurements.

But what it did record is the time it took for the Row Hammer test to complete each of 14 passes of the Row Hammer test alone (the fifteenth was aborted).

The interesting part is that it exactly replicates my own one-step speed change result using 8 passes of the entire memtest battery:

The speed halves after the first Row Hammer pass (within 1 minute), and then remains the same through all subsequent passes. I haven't received an explanation of this, and I don't have any idea what could cause it.

But the fact that it was consistent on two machines, one running the entire battery of tests for 8 passes, and the other running just the Row Hammer for 14 passes seems to me to rule out a
gradual, progressive slowing of the processing time.

I edited the log to show just the date/time stamps of the beginning of each pass, to make it more readable and smaller (still 56K, though).

I've tried to attach it as an odt and as a txt file, but the forum refuses both, so I'll try pasting it in.

I've kept the original log intact, if you'd care to have a copy by PM. My comments are in brackets [].

CPU speed: 2494.2MHz
L1 cache speed: 41135 MB/s
L2 cache speed: 12439 MB/s
L3 cache speed: 9408 MB/s
mem speed: 5559 MB/s
Memory latency: 65.684 ns
CPUSpeedTurboTheoreticalMax = 3092901
Available Memory: 0x3F3567000 (15GB)
Intel E3 Haswell-E chipset init
ECC polling disabled
This platform has 4 logical processors of which 4 are enabled.
TEST SESSION - 2015-08-28 14:36:06
2015-08-28 14:36:06 - CPU selection mode = 0
2015-08-28 14:36:06 - Starting pass #1
2015-08-28 14:36:06 - Running test #13 (Test 13 [Hammer test])
Enabling memory cache for test
2015-08-28 14:36:06 - Locking all memory ranges first...
2015-08-28 14:36:07 - Memory range locked: 0x100000000 - 0x41E500000 (1024KB of available memory left)
2015-08-28 17:04:58 - Finished Stage 2, Pattern=FFFFFFFF
2015-08-28 17:04:58 - Cleanup - Unlocking all memory ranges...
2015-08-28 17:04:58 - MtSupportRunAllTests - Test execution time: 8931.303
2015-08-28 17:04:58 - Starting pass #2
2015-08-28 17:04:58 - Running test #13 (Test 13 [Hammer test])
2015-08-28 21:50:42 - Starting pass #3
2015-08-29 02:36:26 - Starting pass #4
2015-08-29 07:22:10 - Starting pass #5
2015-08-29 12:07:55 - Starting pass #6
2015-08-29 16:53:39 - Starting pass #7
2015-08-29 21:39:24 - Starting pass #8
2015-08-30 02:25:08 - Starting pass #9
2015-08-30 07:10:52 - Starting pass #10
2015-08-30 11:56:35 - Starting pass #11

2015-08-30 16:42:19 - Starting pass #12
2015-08-30 21:28:01 - Starting pass #13
2015-08-31 02:13:45 - Starting pass #14
2015-08-31 06:59:30 - Starting pass #15
2015-08-31 08:51:51 - MtSupportRunAllTests - Test execution time: 6740.607
2015-08-31 08:51:51 - Test aborted


[Row Hammer test times for first (66 hour Row Hammer only) test session: Pass 1=149 min.,Pass 2=286 min., Pass 3=286 min., Pass 4=286 min.,Pass 5=285 min., Pass 6=286 min., Pass 7=285 min.,Pass 8=285 min., Pass 9=286 min., Pass 10=286 min., Pass 12=286 min., Pass 13=286 min, Pass 14=286 min., Pass 15 aborted after 112 min.]
otropogo@gmail.com facebook.com/otropogo

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#36 Post by otropogo »

Scooby wrote:
otropogo wrote: Whom are you quoting?
http://googleprojectzero.blogspot.fr/20 ... -gain.html

I actually got the quote from a pdf that I was unable to find however
the text is the same, you can check for your self

otropogo wrote: Kingston tells me that in order to use the Row Hammer exploit to hack a system, you have to have physical access to the PC or server.

Your quotes above seem to suggest attacks via an internet connection are possible.
http://arxiv.org/abs/1507.06955

http://arxiv.org/pdf/1507.06955v2.pdf



"In this paper we present Rowhammer.js, a JavaScript-based
implementation of the Rowhammer attack. Our attack uses an eviction
strategy found by a generic algorithm that improves the eviction rate
compared to existing eviction strategies from 95.2% to 99.99%.
Rowhammer.js is the first remote software-induced hardware-fault attack.
In contrast to other fault attacks it does not require physical access to
the machine
, or the execution of native code or access to special
instructions. As JavaScript-based fault attacks can be performed on
millions of users stealthily and simultaneously, we propose
countermeasures that can be implemented immediately. "

Seems like Kingston is lying to you!?
Thanks for the references. Perhaps my informant at Kingston isn't as well informed as I thought. Or, maybe I misunderstood him. We had a very long telephone conversation, and such things are better discussed in writing, IMO.

But if you look at the log I pasted in the previous post, you may notice the following stats:

mem speed: 5559 MB/s
Memory latency: 65.684 ns

When I ran the memtest on what I was told was the very 16GB SODIMM kit (ie. not just the same model, but the actual kit sent to me) reported in the text log, these figures were shown as:

total physical memory: 16435M (18314 MB/s)
Memory Latency: 25.224 ns

That doesn't look like the same SODIMM set to me...

What do you think?

Given that RAM is indispensable to computing, I hate to think that the vendors' senior technical staff are either incompetent or dishonest, let alone both. But if I have to come to that conclusion, what to do?

Do you have a suggestion?

If anyone out that has a benign explanation, including that I've misunderstood the terms, am comparing apples and oranges, etc., please speak up.
otropogo@gmail.com facebook.com/otropogo

Bindee

#37 Post by Bindee »

Crossed wires.

I meant when you start from test #1 you are using a different memory address pattern than the constant row hammering of #13

Unlike the other tests #13 hammers across a entire single line of addresses

So in my opinion it should be done separately because the constant hammering refresh of the individual cells is different but the ram will still have to deal with the leaked data from #13 back to #1 so it will be slow in correcting and refreshing as the capacitors are swung into a middle state and can still leak but the other tests will not hammer it the same.

But if you unplug the computer and turn it off and on again to physically clear the ram then the first run will be fast again as the capacitors will have been reset between the cells as the damage is not permanent.

Everyones ram will slow down as the hammering is forcing data to leak between the memory cells capacitors to temporarily corrupt the surrounding cells so it all depends for how long the ram can flip the cells correctly before it gets flooded to the point it can no longer correct itself and the transistors don't know if they should be in a positive or negative state due to the change in capacitance.

Some sticks of ram will corrupt on a single pass others will corrupt after hours of hammering hence why they don't really know how many sticks are affected.

Some feel that all DDR3 will leak if you hammer it for long enough but no one leaves their computer locked up for hours so they are unsure yet how how effective webpages running a java exploit will be at taking over a PC.

User avatar
otropogo
Posts: 764
Joined: Sat 24 Oct 2009, 15:17
Location: Montreal
Contact:

#38 Post by otropogo »

Kingston HyperX RAM-Row Hammer update.

I haven't had the hoped for explanation from Kingston regarding slowing of the Memtest with repetition. So this remains a mystery to me.

I did return my original HyperX SODIMM set and the second HyperX kit sent to me, keeping the standard speed kit that passed the tests cited above. Kingston prepaid Fedex shipping, so my only expense was driving to the Fedex centre to deliver the package and complete the air bill.

Thanks to GCMartin for bringing this problem to my attention, to Bindee for showing me how to test for it, and to Kingston for fixing it.

Since I was the first customer to complain to them about this issue, Kingston should be able address it much more efficiently for subsequent callers.
otropogo@gmail.com facebook.com/otropogo

gcmartin

Virus which does GOOD on your system?

#39 Post by gcmartin »

This virus should get some interest in this forum, especially in light of how something that is bad for your system is actually good for your system.

Released as a Virus attacking your router to make each router better

You be the judge

Post Reply