Wow
The programming effort needed to do all that must be immense - or is all plug in modules these days?
I hope it's all been thoroughly tested....
Alan
Storage startup WekaIO punts latency-slashing parallel file system tech
Storage startup WekaIO has joined a growing crowd of techies making large performance gains and latency lowering file system tech moves. It has claimed to have developed the highest-performance, lowest-latency file system ever created – which is quite a bold statement – but since this technology area is developing so rapidly, …
-
Saturday 23rd December 2017 07:02 GMT Anonymous Coward
Very interesting if it works though some details in the article make me flinch:
Software efficiency measured as 5 percent of what? Without telling us how big the server is, the 5% figure is meaningless (80 core & half a TB RAM?).
What do you mean it only uses the resources allocated to it? I'd expect to see a graph that shows the latency/throughput that MatrixFS can provide based on the amount of resources dedicated to it.
Performance of find and ls?! I don't see anyone using linux command line utilities like find or ls as a part of any kind of high performance real world application. If you are using them, then 2 hours vs 6 hours would both be unacceptable and they would have rewritten their application.
Some of the above don't take away from what they have built, it just doesn't help for credibility. I want to see scale and numbers for real world applications (with customer names) and a reference architecture in standard HCI systems (e.g. Nutanix) with a resources vs performance graph. I personally doubt they will ever replace the appliances out there but when you pair it with HCI, I can see their value for certain applications.
-
Thursday 28th December 2017 19:56 GMT Richard 12
Can't be an RTOS either
An RTOS cannot exist in the user space of a not-RTOS kernel, by fundamental definition - the kernel is deciding the timeslicing all user space applications get, thus no application can ever make any of the timing guarantees required of an RTOS.
- The kernel can decide not to honour any timer it wishes, though in practice the Linux millisecond timer is very good.
Either the kernel is an RTOS, or nothing is. It might however be very fast and efficient.
-
Saturday 30th December 2017 07:43 GMT CheesyTheClown
Re: Can't be an RTOS either
I remember as a kid, a school teacher challenged me and some friends to cut a hole in a business card large enough to walk through. After 10 minutes we were walking up to the teacher to explain it couldn’t be done. Then a friend turned heal and went back and made a series of cuts and we walked through the hole.
This puzzle allowed me to be the kid with the scissors.
A real-time OS needs to handle “interrupts” as they come in. Code in an RTOS should never block and if disk access is required, the program should send a read request as a queued event to the disk I/O subsystem and receive the result back once it is finished. Compared to when I was coding for QNX in the 90s, this type of programming is extremely easy thanks to improved language support with extensions like lambda functions.
Now, the “time” aspect of RTOS generally would mean that as soon as a timer interrupt fires at a specific interval on the system’s programmable hardware timer, a deterministic number of cycles should pass before the entry point to the handler. This allowed “Real Time”.
If we eliminate the time aspect of the processes involved... which a storage system wouldn’t be concerned with anyway, it is possible to develop an operating system that behaves as an RTOS if “interrupt” handling can be made deterministic. In addition, if all code is written as non-blocking by handling all protocols using lambdas and “async programming patterns”. So, as opposed to a general purpose OS which task switches without reason, an RTOS would schedule everything based on prioritized event queues.
So, the main issue is how to process interrupts in real time within a user-space process.
1) Implement interrupt priority for waking the scheduler of the process on incoming interrupt. This can be done in a few creative ways, a kernel module is one method, alternatively setting processor affinity, to reserve 100% of a CPU core and running a spin-lock sleep could work. There are dozens of ways to do this.
2) Expose a second level MMU to the app allowing the process to directly handle protection faults. This is the obvious method of providing deterministic interrupt handling.
3) Expose the virtual NIC as a PCIe device to the app. Then when the app is managing its own MMU with it’s own GDt and IDT, the PCIe adapter can trigger faults for MMIO interrupts within application managed memory space. So as the RTOS app sets virtual protection on an Ethernet memory address, it should signal the app in a reasonably “real-time” fashion.
So, while I was with you all the way up to writing a full page of text... I got to just in front of the teacher’s desk before turning back to cut a whole in a business card to walk through. As long as the events aren’t actually timer triggered, I believe an RTOS in user space via hardware virtualization is entirely reasonable. :)
-
Monday 8th January 2018 07:52 GMT Richard 12
Re: Can't be an RTOS either
What a wall of nonsense...
An RTOS makes guarantees about timing. That's what makes it an RTOS.
A "normal" kernel does not make guarantees about timing. It is "best effort" and you'll get your turn eventually.
Claiming RTOS timing guarantees in user space is making a claim of CPU time that you don't have control over. Like you saying you can walk through a business card, but teacher takes away the scissors and the business card, before you can attempt it.
He'll give it back to you tomorrow.
-
-
Biting the hand that feeds IT
