Worth mentioning: this might work some of the time, but in an asynchronous (e.g. real) network, it could be unsafe. The described locking scheme does not actually ensure no two processes hold the lock at the same time. Even if it did, it would not ensure that side effects, like writing to block storage, would be safe. Martin Kleppmann has a terrific overview of why “distributed locks” generally don’t do what people think, and what to do instead: https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html
Thanks for this feedback. Actually my design already takes this problem into account.
Kleppmann warns that a process can freeze for an arbitrary amount of time due to garbage collection, network problems, CPU starvation, etc. I mitigate this in my design through two ways:
By recommending a long TTL, in the order of minutes. Kleppmann cited a problem at Github where packets were delayed for 90 seconds. The default TTL I recommend is 5 minutes.
By refreshing the lock from time to time, and by checking for its health periodically. The fresh- and healthcheck interval must be sufficiently short as to prevent the very problems described by Kleppmann.
It’s described in section “Long-running operations”.
Kleppmann proposes the use of fencing tokens, which indeed guarantees that such problems don’t occur, but it requires sufficient support by all systems. My design doesn’t provide as strong of a guarantee, but we can make it arbitrarily approach 100% safety by configuring the various timing settings (TTL, refresh interval).
I’m working on a Ruby app server that’s 2x-6x faster than Puma, Unicorn and Phusion Passenger. Lots of C++, profling, optimizing malloc calls, optimizing system call usage, zero-copy techniques, CPU cache and pipeline optimizations, etc.
I’m trying to stimulate contributions to my open source project, Phusion Passenger (https://github.com/phusion/passenger). It’s an app server for Ruby, Python and Node.js, but it has historically had very few contributors because Passenger is written mostly in C++, making the contribution barrier high for the target group.
It’s a bit sad that in 2014, ZFS is still the only filesystem that checksums data (as opposed to metadata only). I am worried that my backup disk is becoming increasingly corrupted without me knowing.
Not to be too pedantic, but btrfs and nilfs also checksum data. There’s also a comparison of filesystems and their features here. I don’t recognize most of the file systems that support it, and I think that fundamentally it’s because checksumming is something you don’t need your file system to do. It’s fine to just do it yourself if your data matters that much. In fact, for most file systems, it’s just as good to have someone just manually figure out your checksum is broken, because there’s nothing you can do about it!
What makes zfs and btrfs thrilling is that they maintain all of these snapshot versions, so if they detect data corruption, they can do self-healing by comparing against other snapshots. Snapshotting is a relatively new feature for filesystems, so it makes sense that checksumming is too.
The other thing is that it’s difficult to add new features to a file system, so people often do it by just making an entirely new version of the file system, so it’s fairly straightforward why only the newest file systems have this new feature.
While these specific drivers aren’t allowed to be redistributed by Google, none of the other drivers are allowed to be redistributed by ROM developers, either. Nobody seems to care about that and does it anyway.
While I understand his point of being powerless in the face of binary drivers, couldn’t he have architected the graphics system in such a way that that’s ok? I believe Windows Vista and later implements a graphics system just restarts the graphics driver if it crashes, instead of letting it take down the OS. The ability to be able to not get affected by someone elses bugs and to be able to point fingers is great.
Docker is wonderful. I’m using it to build a continuous integration system that can run my unit tests as root. It’s completely isolated and it uses almost no resources, compared to spawning up an entire VM. There are still some bugs and security issues here and there but they’re coming along quickly.
Finally a sane Linux answer to FreeBSD jails. It’s about time.
I would like to see more SELinux documentation for developers. As an app developer how do I write a policy and distribute it to users? Right now it’s very hard to get the mental model right. All the documentation I’ve read so far is either large and complicated or overly simplistic. It would be great if there’s documentation that explains things step by step.
Unfortunately I have to recommend my users to disable SELinux at this time. I’ve been asking for SELinux compatibility contributions for years. Some people volunteered but never finished the job. It seems either very few people understand SELinux, or it’s so much of a pain no contributor wants to bother.
Worth mentioning: this might work some of the time, but in an asynchronous (e.g. real) network, it could be unsafe. The described locking scheme does not actually ensure no two processes hold the lock at the same time. Even if it did, it would not ensure that side effects, like writing to block storage, would be safe. Martin Kleppmann has a terrific overview of why “distributed locks” generally don’t do what people think, and what to do instead: https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html
Thanks for this feedback. Actually my design already takes this problem into account.
Kleppmann warns that a process can freeze for an arbitrary amount of time due to garbage collection, network problems, CPU starvation, etc. I mitigate this in my design through two ways:
It’s described in section “Long-running operations”.
Kleppmann proposes the use of fencing tokens, which indeed guarantees that such problems don’t occur, but it requires sufficient support by all systems. My design doesn’t provide as strong of a guarantee, but we can make it arbitrarily approach 100% safety by configuring the various timing settings (TTL, refresh interval).
I’m working on a Ruby app server that’s 2x-6x faster than Puma, Unicorn and Phusion Passenger. Lots of C++, profling, optimizing malloc calls, optimizing system call usage, zero-copy techniques, CPU cache and pipeline optimizations, etc.
I’m trying to stimulate contributions to my open source project, Phusion Passenger (https://github.com/phusion/passenger). It’s an app server for Ruby, Python and Node.js, but it has historically had very few contributors because Passenger is written mostly in C++, making the contribution barrier high for the target group.
In the last 2 weeks, I’ve been writing developer documentation and making code walkthrough screencasts to help new contributors (http://blog.phusion.nl/2014/06/16/announcing-new-revamped-developer-documentation/).
Today, I’ve started working on a marketing campaign and reward system for encouraging people to contribute. The specification is here: https://github.com/phusion/passenger/wiki/Contribution-Campaign
It is super hard to get Rubyists to write C. It’s one of the big reasons Shoes moved to JRuby. I feel your pain :(
This time I’m also enlisting the help from Node.js and Meteor devs. Not sure whether the situation is better there, but one can hope. :)
It’s a bit sad that in 2014, ZFS is still the only filesystem that checksums data (as opposed to metadata only). I am worried that my backup disk is becoming increasingly corrupted without me knowing.
Not to be too pedantic, but btrfs and nilfs also checksum data. There’s also a comparison of filesystems and their features here. I don’t recognize most of the file systems that support it, and I think that fundamentally it’s because checksumming is something you don’t need your file system to do. It’s fine to just do it yourself if your data matters that much. In fact, for most file systems, it’s just as good to have someone just manually figure out your checksum is broken, because there’s nothing you can do about it!
What makes zfs and btrfs thrilling is that they maintain all of these snapshot versions, so if they detect data corruption, they can do self-healing by comparing against other snapshots. Snapshotting is a relatively new feature for filesystems, so it makes sense that checksumming is too.
The other thing is that it’s difficult to add new features to a file system, so people often do it by just making an entirely new version of the file system, so it’s fairly straightforward why only the newest file systems have this new feature.
While these specific drivers aren’t allowed to be redistributed by Google, none of the other drivers are allowed to be redistributed by ROM developers, either. Nobody seems to care about that and does it anyway.
I wrote about this 2 years ago while I had my own Android ROM.
While I understand his point of being powerless in the face of binary drivers, couldn’t he have architected the graphics system in such a way that that’s ok? I believe Windows Vista and later implements a graphics system just restarts the graphics driver if it crashes, instead of letting it take down the OS. The ability to be able to not get affected by someone elses bugs and to be able to point fingers is great.
Docker is wonderful. I’m using it to build a continuous integration system that can run my unit tests as root. It’s completely isolated and it uses almost no resources, compared to spawning up an entire VM. There are still some bugs and security issues here and there but they’re coming along quickly.
Finally a sane Linux answer to FreeBSD jails. It’s about time.
I would like to see more SELinux documentation for developers. As an app developer how do I write a policy and distribute it to users? Right now it’s very hard to get the mental model right. All the documentation I’ve read so far is either large and complicated or overly simplistic. It would be great if there’s documentation that explains things step by step.
Unfortunately I have to recommend my users to disable SELinux at this time. I’ve been asking for SELinux compatibility contributions for years. Some people volunteered but never finished the job. It seems either very few people understand SELinux, or it’s so much of a pain no contributor wants to bother.