RamDisk

so way back when ach was stringing us on with promises of a new computer, me and pat talked about how big games are. by extension, this makes load times obnoxious. capacity and speed are on opposite sides of the same spectrum for typical HDD's and ssd's are a different tradeoff but not a more practical one (capacity and speed Vs. cost).

i just found a cool piece of software that does something that we sorta-but-not-really talked about in our posts. it makes a drive out of RAM. RAM while being super fast is limited by both capacity and the fact that when it loses power it loses all info stored within it.

this software combines RAM and some sort of persistent drive (HDD and SSD) to make a RAM drive that won't lose info on power down. instead info gets loaded on boot into the RAM drive and then pushed back onto the HDD/SSD on power down. i can only think of a few practical uses of this -- by writing to RAM during power on and the drive only on power off you could extend the life of an SSD considerably if you had an app that did very frequent writes. this would also be good for an app that does a lot of reads without loading what its reading into memory

the 2nd part lead me to a fun not-so-practical-but-still-kinda-cool idea. you could make a RAM drive and store an entire game on it. as long as you had enough RAM left over to run the game you would have almost non-existant startup, load, save, anything-at-all times

i doubt any of us have a good enough computer to try this with something like skyrim (would probably take around 24gb of ram) but one of you should try with something smaller (that still has load times that are mostly due to disk read, wc3 doesnt count)
here's the software
http://www.ltr-data.se/opencode.html/

besides this for fun experiment, it's actually a good piece of software. you can allocate a RAM drive of, say, 4gb and if you only take up 10mb of it then it will only reserve 10mb of your system's RAM

here's a shot of its UI that i stole from somewhere

Back when we played swtor, i was looking for a way to decrease load times cause for some reason it takes like 3+ mins to load certain areas on my computer, and i came across this: http://www.swtor.com/community/showthread.php?t=394951
Which is why i was saying you could load games into ram in my post when we were talking about it, though i was saying to do it directly rather then having it separated like this program does where theres 2 copies (1 in ram and 1 in ramdisk)

I didnt post about the swtor thing back then because i actually forgot that specific post until today when i read "ramdisk" which triggered the memory of that swtor post, i was just vaguely remembering seeing something that could do something like i was thinking of somewhere

after looking into it a bit more, mostly going off of that link, it's actually a lot more cool than i thought. it does some things i didnt think were possible
the reason i said load a full game onto ram AND run a full game on ram was because i couldnt think of how it could work well otherwise. like how it could install in one location and then know to pull resource files from other locations during run time. but apparently they thought of this. the solution runs off of something called "Junction Points." i think the basic idea of them is that they overwrite some file locations. So say you have a big complicated game that has a few folders of resources (ie textures, animations etc). Load times are mostly just the game going into the (relatively super slow) hard drive and pulling out the relevant resources to display the following screens until the next load time; the rest of the things that happen in a load time happen a good bit quicker. So if you were to RAMdisk your resource files, like the link pat posted, it would create a junction point. When a load happens it would go looking in the partitioned ram rather than in the default install directory. To give you an idea, hdd's have a read speed of around 600mb/s, while ram standard ram is something like 60gb/s. thats 100x faster (i hope that math is right).

the post pat linked also brings up in game performance increases. animations aren't loaded all at once, usually. usually they're loaded as they're needed -- sometimes causing framerate drops. if you put the animations into ram via the same process you'll effectively beat it to the punch. if you've ever played sc2, this would like a less heavy handed "unit preloader" map

thats awesome.

Do you think this might end up being a common place usage of ram in the distant future?

I feel like it would make sense if the capabilities of ram are outstripping application/game needs, like you guys were talking about earlier, then it would make sense to start making an excess of ram useful.

This is really interesting. I've only got 16GB, so it wouldn't work with a monster of a game like Skyrim.

I may try with a smaller game; I'll post here how it works.

Its not the distant future, the amount of ram computers can have is already way higher then applications and games need,
32gb is all you would need to do this for all games of today, using that program at least, and thats not too hard to get

You can get even more then that right now, 128gb of ram is possible with this board:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813130681
Although currently finding 16gb sticks is difficult, i actually cant find any, im sure you can get them soon though since they are selling boards with the ability to go up to 128gb, but you would need to fill all 8 ram slots in that motherboard with a 16gb stick all of the same type and you would have as much ram as SSDs have storage

I wouldnt recommend doing that though, since the ram timing on 16gb sticks will probably be very slow compared to smaller sticks



But i was talking about combining ram and storage which would be better (takes up less total space and wouldnt need any configuration) then using something like that program, and that should be possible within 10ish years
Though the current system is working pretty well right now and what im talking about would require a redesign of how ram works, so while it would be possible within 10ish years, i dont think it will exist in 10ish years

what i meant by 'common place,' wasn't whether it'd be useful in the distant future, but whether it would be standard pre-purchase.
the same way microsoft word is standard, cus theres no reason you'd ever get a computer and be like "oh i dont want to be able to edit text documents"

Oh, no, a program like that probably wont become a standard like word

Well, it should be.
Like seren said, ram is 100x (ish) faster than HDD, so i dont see why this shouldn't be a direction technology movies, and not just remain a 3rd party thing that people with know-how can do.

joe wrote:

This is really interesting. I've only got 16GB, so it wouldn't work with a monster of a game like Skyrim.

the beauty of junction points is that you don't need to fit the full thing into ram, just part of it. so you could add a junction point for all the textures of skyrim. this would be a lot more reasonable (and totally doable with 16gb) than adding the full game.

joe wrote:

I may try with a smaller game; I'll post here how it works.

yeah definitely let us know, ive been curious

ach wrote:

Do you think this might end up being a common place usage of ram in the distant future?

well yes and no. like pat said it probably wont ever be something like word, it's a bit complicated for that
but i can definitely see it becoming common place among gamers and media professionals
currently RAM is developing at a much faster pace than the requirements for RAM are. on the other hand, the biggest increase in any spec req for games is disk space. since disks are getting minimally faster but resource files continue to get much larger this means less ideal performance for games. it just makes a lot of sense to supplement with something like ramdisk
some disk drives are shipping with ram disk software. currently it's only enthusiast ssd drives, but thats how everything starts
http://www.newegg.com/Product/Product.aspx?Item=20-785-002
here's a pretty descriptive page on how their version of ramdisk works / the perks of it
http://rog.asus.com/technology/republic-of-gamers-motherboard-innovations/ramdisk/

We might be looking at Non-Volatile RAM in the future, as it allows persistent data storage with the performance of standard RAM.

not even all that far
intel's next generation of processors (6th gen) is going to feature support for ddr4 ram (on high level chips only). ddr4 is an improvement over ddr3 in all ways save material cost, but even that is only initial. it's believed that ddr4 will be the last generation of d-ram. i also believe that -- there just wont be any practical point to get to ddr5 ram for d-ram
from there things will expand in at least one direction. the current top candidates are mram and rram
spec wise, they both are, technically, better than dram
http://www.pcworld.com/article/2084240/new-types-of-ram-could-revolutionize-your-pc.html
they do what you just described, oreo, but that has its own tradeoffs. one of them is described in the article. personally i like the encapsulation of volatile vs non volatile. even once hardware gets to the point that everything could be non-volatile and still instantaneously fast i would still want, from a software development pov, access to memory that clears on power off. being able to assume your initial state and ignore your final state is a very, very, very powerful thing
you could theoretically achieve the same thing with software, but software emulating hardware functionality goes terribly (historically). this is because you need a lot of people to work together and people don't do that when they think their way is the best way. ideally the new ram would have a dynamically sized partition between volatile and non volatile. that would be awesome but it sounds too good to be possible

i would rather gpu and ram be combined and cpu and gpu be combined (each taking separate components of the gpu) to replicate what the gpu currently does, but only once there wouldnt be much of a loss

Serenity09 wrote:

even once hardware gets to the point that everything could be non-volatile and still instantaneously fast i would still want, from a software development pov, access to memory that clears on power off. being able to assume your initial state and ignore your final state is a very, very, very powerful thing

If they havent run your program before the initial state would be the same as it is with volatile ram and you would assume your initial state like normal without any issues, if they have run your program before then its just like they never stopped running the program and there is no initial state so it would just resume from where it last was when they start it again

If you dont want your program to resume as if it never stopped you can just have it clear itself (or just the parts you dont want to keep in memory which would be more efficient) from memory when the user quits or when w/e task the program was doing is finished so you can still ignore the final state if you need to, and if you dont need to then the final state doesnt matter since when the program starts it will just resume from where it was last time

The only time the final state would be an issue is if something goes wrong with the program, like an infinite loop or some other bug that causes it to get stuck in some way that makes it unusable, for that im assuming that the OSs of that time that supports that kind of memory would have a way to forcibly remove a program from memory like how you can force a program to close which would solve that issue and the next time the user starts the program it would be in the initial state again (it would still load settings since you would still need to store critical things like settings on the harddrive, but doing that is basically the same as restarting a program right now)
I dont think that type of memory would work if there was no way for the user to force a program to clear what its stored in memory

The OS would need a way to detect issues at start up with its own critical systems so that it can clear those if it needs to and if it cant fix them then it just clears everything at startup, it would also need a way for the user to flag critical systems to be cleared on next start up because sometimes something important fails but your computer could still start and the OS probably wouldnt detect something like that (you would need to do it at start up because if you do it at shut down and the computer loses power in the middle of it you would need to do it at start up anyway so might as well just do it at start up and save the extra potential step)


So yeah, it wouldnt be an issue unless youre an OS dev or if the OSs of that time are really poorly made
A terrible OS is actually a pretty high possibility since its all new so at the start i would want both types of ram just because i dont trust microsoft or apple to be competent enough to get it right with the first version of their OS that supports this type of ram

that was a really elaborate way to demonstrate a software solution but i never said they werent any just that everyone will always have their own "best" solution and that a hardware solution was nice because it avoided that + all the, usually unexpected, side effects that can have
im not gonna get into all your slight tangents

have you heard of the halting problem? perfect infinite loop detection isnt possible and even very good infinite loop detection, like what we have, is pretty hard

Serenity09 wrote:

that was a really elaborate way to demonstrate a software solution but i never said they werent any just that everyone will always have their own "best" solution and that a hardware solution was nice because it avoided that + all the, usually unexpected, side effects that can have

Even with both types of ram the non volatile could still get something loaded into it that causes problems that can only be solved by clearing it like i was saying so that hardware solution doesnt even solve the problem, unless you never intend to use the non volatile ram

I cant think of a good hardware solution, once you start using non-volatile memory this kind of thing inherently becomes a problem
The software solution i came up with works best for both programmers and normal users and isnt even complicated, your just clearing memory for when something goes wrong in memory (or cleaning up memory when you dont want something in there all the time), its no different from restoring default settings when a config file gets screwed up, you would program in an option to restore default settings in a program that relied on settings right, so you would do the same for clearing ram if your program relies on a certain initial state

Serenity09 wrote:

have you heard of the halting problem? perfect infinite loop detection isnt possible and even very good infinite loop detection, like what we have, is pretty hard

If the OS fails completely you can just clear all memory from within the bios so if its stuck in a loop that it cant recover from or detect at all due to a memory issue you just press 1 of the function keys at post and select clear ram in that menu (hopefully by that time you can use the bios to connect to and dl from the internet so you can verify the integrity of your OS and dl anything that it needs to fix itself if something other than memory goes horribly wrong (similar to steams system where you can verify your game files))

I assume a motherboard that supports non volatile ram would have an option like that, because again non volatile ram wouldnt work without the ability to clear it when something goes terribly wrong

Serious problems like that shouldnt happen too often though, no more often than your OS screws up now, so you would rarely need to use it
We would just format our ram every now and then just like how today we format a hard drive every now and then, and like formatting a hard drive it should be a last resort, after everything else fails, so the detection would need to be in there even if it cant catch everything because it can catch some things

Quote :

Even with both types of ram the non volatile could still get something loaded into it that causes problems that can only be solved by clearing it like i was saying so that hardware solution doesnt even solve the problem, unless you never intend to use the non volatile ram

i said that the volatile property was a very valuable piece of encapsulation and it works best on a hardware level. i said that it was valuable -- a simple hardware solution is waaay better than complicated and glitchy each-their-own software software solution -- but i didnt say anything about it being worth or not worth the tradeoffs once other ram was an option. given the choice between having to choose what class of ram i load into whenever i load anything vs having it handled entirely by hardware and OS, i would probably pick the 2nd. but even then it would be nice to have an override to load into dynamically sized volatile ram. most programs will work well / better continuously, but a lot of programs just are fundamentally meant to be run every once in a while by, ideally, volatile memory
the software solution you came up with isnt even psuedocode. it's not a software solution, it's a slightly educated idea that relies on, what are realistically, oversimplifications. it will have its own tradeoffs

Serenity09 wrote:

i said that the volatile property was a very valuable piece of encapsulation and it works best on a hardware level. i said that it was valuable -- a simple hardware solution is waaay better than complicated and glitchy each-their-own software software solution -- but i didnt say anything about it being worth or not worth the tradeoffs once other ram was an option. given the choice between having to choose what class of ram i load into whenever i load anything vs having it handled entirely by hardware and OS, i would probably pick the 2nd. but even then it would be nice to have an override to load into dynamically sized volatile ram. most programs will work well / better continuously, but a lot of programs just are fundamentally meant to be run every once in a while by, ideally, volatile memory
the software solution you came up with isnt even psuedocode. it's not a software solution, it's a slightly educated idea that relies on, what are realistically, oversimplifications. it will have its own tradeoffs

All i was doing was putting forward an idea to make volatile memory work without the need for non-volatile memory and how it could still be just as powerful as non-volatile memory, i called it a software solution because thats what it would be if it existed, but it doesnt exist so there was no reason to not simplify it down like i did

You're just saying we should always use volatile ram, which is like saying we should just use (insert old technology here) because (insert new technology here) has issues that we havent figured out how to solve yet and rather than trying to solve them we should just keep using the old technology because we know it works
Which is a terrible way to think

Yes a hardware solution to non-volatile ram is better than what i said, but like i said there isnt really a good hardware solution to solve this (without just throwing a few sticks of the old technology in there) and what i said is the best possible potential software solution
We have no way to know if its glitchy or not, we could speculate how glitchy it would be based on what we have today, and if you base it on that it wouldnt be too glitchy since clearing ram is a thing that happens all the time right now and its totally fine

pat wrote:

All i was doing was putting forward an idea to make volatile memory work without the need for non-volatile memory and how it could still be just as powerful as non-volatile memory, i called it a software solution because thats what it would be if it existed, but it doesnt exist so there was no reason to not simplify it down like i did

im gonna assume that you got volatile and non volatile mixed up everywhere here and just go with that. my point was that there would be tradeoffs of its own, not questioning your extensive experience with developing software and hardware related to ram

pat wrote:

You're just saying we should always use volatile ram, which is like saying we should just use (insert old technology here) because (insert new technology here) has issues that we havent figured out how to solve yet and rather than trying to solve them we should just keep using the old technology because we know it works

stop over analyzing just individual parts of what im saying.
i value some aspects of volatile memory; i value some aspects of non volatile memory. forcing a choice, id pick non volatile, but would still enjoy low level access -- when i want it -- to volatile memory. i've said a few times that i'd go with non volatile ram if it was one or the other.
in my initial post on this i said that you could probably have the software solution provide an api for low level access into ram. the library backing the api would utilize some watchable wrapper or global hashset or whatever to emulate the volatile property when wanted

pat wrote:

Yes a hardware solution to non-volatile ram is better than what i said, but like i said there isnt really a good hardware solution to solve this (without just throwing a few sticks of the old technology in there) and what i said is the best possible potential software solution

dude you fix ram you don't engineer it. put some asterixes in there or something

Serenity09 wrote:

im gonna assume that you got volatile and non volatile mixed up everywhere here and just go with that

Yeah i was going to put that first part in a different way after typing it and swapped stuff around and didnt erase all of it so volatile and non volatile got swapped to the wrong places and i didnt read it back closely enough to notice

Serenity09 wrote:

i value some aspects of volatile memory; i value some aspects of non volatile memory. forcing a choice, id pick non volatile, but would still enjoy low level access -- when i want it -- to volatile memory. i've said a few times that i'd go with non volatile ram if it was one or the other.
in my initial post on this i said that you could probably have the software solution provide an api for low level access into ram. the library backing the api would utilize some watchable wrapper or global hashset or whatever to emulate the volatile property when wanted

Thats not what you were saying at first, if that was what you were saying you couldve just agreed with me and added your bit about an api, because that was basically what i was trying to do with what i said, i just didnt go into detail about how it would work, and the part about the OS was so users had some control over it too and to explain that it would need some sort of fail safe in the event of catastrophic failure

At first you said:

Serenity09 wrote:

ideally the new ram would have a dynamically sized partition between volatile and non volatile. that would be awesome but it sounds too good to be possible

I assumed you meant that as a hardware solution, like the ram has a chip on it that handles the dynamic allocation and that it can emulate volatility, which is too good to be possible, so everything i said was a realistic way to emulate volatility by using software without it being too terrible, and that it could still be as powerful as being volatile

I didnt mention dynamic allocation because its pointless when emulating it like i was saying, you would want everything to be loaded into the volatile partition, so may as well make it all capable of being volatile when it needs to be instead of partitioning it

pat you suck, what i said makes sense. rephrased: "ideally, flagging data as volatile would be handled dynamically by hardware, but you could also probably achieve the same effect by emulating that via software. software emulating hardware typically sucks though so having the hardware do it all would be a lot better"

anyways, wrapping around to ramdisk:
with a system that uses non-volatile ram, a ram disk would actually just be a partitioned drive. this would add performance (you wouldnt have to do an initial load into memory). the only downside is that if data becomes corrupted from anything (probably the program itself) then it'll be a bit of a pain. but this is the same scenario you have with current storage drives (hdd, ssd) so not really especially noteworthy