1. 3

    Just this one so far: Designing Data-Intensive Applications

    I do have a tendency to get about 50 pages through most books that I start, and then fail to pick them back up. It’s actually my new year’s resolution to finish a bunch of the books that I started in 2017… just haven’t figured out which ones yet.

    1. 4

      In addition to what skade said, I also think it’s not a bad idea to read books “breadth-first”. That is, scan over the table of contents, then pick something you like. And even skim that chapter, pick up some of the key words and ideas, and come back to it later when you feel a need to (e.g. you need to implement something).

      Some people express guilt over not finishing books, but I don’t feel bad about that at all. As long as I can finish SOME book, I don’t feel bad about not finishing a particular book. Particularly technical books. Actually I’m pretty sure I always read tech books in a random access “need to know” fashion, so I can’t say I’ve “finished” any of them.

      1. 3

        Throw them on a stash randomly, read them top to bottom. If you feel like dropping one, drop it for good.

        1. 2

          That’s not a bad idea! Having to prioritize fuels the procrastination. I’ll give this a shot.

      1. 3

        I’ve used Etherpad a lot, since it’s the primary way various openstack projects take notes at the meetups. Is Hackpad a fork of Etherpad? Anyone who’s used both care to comment on the differences?

        1. 4

          I don’t know if it’s a fork, but the Readme claims that it’s at least based on Etherpad:

          Hackpad is a web-based realtime wiki, based on the open source EtherPad collaborative document editor.

        1. 1

          Looks like a nice music player. Is it purely streaming via bittorrent, or can I point it at my local music library too?

          1. 3

            It will play your local library.

          1. 3

            The mozilla blog has some more info too. Most concerning to me is:

            … in all the user directories it can access it looks for .bash_history, .mysql_history, .pgsql_history, .ssh configuration files and keys …

            Guess I’ll be rotating SSH keys soon. The post also mentions that Windows and Linux users are vulnerable (but not OSX). I wonder if BSD is affected.

            1. 4

              It mentions that OSX is vulnerable, but that the found exploit didn’t target it:

              Mac users are not targeted by this particular exploit but would not be immune should someone create a different payload.

            1. 2

              FTR, Openshift is the best free hosting solution I’ve found so far, definitely recommend.

              1. 2

                How would you compare it to other PaaS services (say, Heroku)?

              1. 2

                Very interesting. I’ve tried stumpwm in the past, which seems to have similar design goals. I think the split here is that EXWM goes a step further and regards individual windows as buffers, where stumpwm uses workspaces (or “frames”) as buffers. I’ll have to give this a shot when it’s a little more complete.

                1. 6

                  I honestly had no idea that you could run Nix on OSX; Neat! I may try to use this to help make my linux and OSX environments much more similar.

                  1. 3

                    I ran in to issues with software not building quite a bit. I am not sure of the current state though.

                    1. 3

                      It can be hit or miss most of the time But if more people use it and help it should improve. I’m somewhat in mid stream replacing homebrew with nix on osx. Still can’t get rid of homebrew just yet but getting there slowly.

                  1. 1

                    Very interesting look! Being mostly entrenched in the openstack world, I’d not heard of Ganeti before. It looks a little simpler than openstack, and I may look into it the next time I’m building a “cloud” from scratch.

                    1. 2

                      You might also be interested in Synnefo, which adds an OpenStack-compatible API and some other stuff to Ganeti.

                    1. 5

                      It makes me happy to see things like this succeed :-) Good for Ola**2. I’ve found that the the Django Girls Tutorial referenced in that article is much more approachable than the official one.

                      1. 6

                        I think it should be highlighted that suckless is writing their own init system. However, when someone is writing a “competing” tool, it doesn’t inspire much confidence in the argument (or the software, really) when their criticism of the other tool starts off like this:

                        There is a menace which is spreading like a disease throughout the Linux world, it is called systemd.

                        Think you can write a better init system? Great. Just ship your code, and enforce its superiority through correct design decisions. Let me know when a bunch of distributions have decided sinit is better than systemd (or even SysVInit, for that matter).

                        1. 11

                          While it’s not an excuse for inflammatory language, keep in mind that this tired debate is basically hijacked by very loud, very obnoxious voices. So it’s understandable that some people who dislike systemd might get angry at constant accusations of extreme conservatism and incompetence. Angry enough to not being able to present their criticism without that anger showing. Same goes for pro-systemd folks.

                          Also:

                          I think it should be highlighted that suckless is writing their own init system.

                          uwotm8. A dumb init isn’t an init system, even by very lax definitions of such a thing. sinit is an init, it’s dumb as a brick and in no way is it enough to boot a system to a useable state.

                          It’s not completely honest, btw, to invoke the pure technical merit argument. Software rarely rises on pure technical merit, there’s a lot of politics (not talking evil conspiracies here) in the opensource community.

                          1. 7

                            sinit, like most other suckless projects, does a single thing well, simply, and rests on the laurels of the Unix philosophy.

                            1. 2

                              I’m not criticizing sinit here. Just being anal about widely used (even relatively ambigious) terms.

                              1. 3

                                I know. I was just making the simplest argument, without inflammatory language, for sinit.

                                I’m not at all sure why more arguments systemd v. any other init system don’t just boil down to the statement I just made. But, they don’t; Because people are terrible.

                        1. 2

                          I use openafs, which is unsigned. Pain in the bosom.

                          1. 2

                            Same here… the cost for signing is probably more than the openafs project will ever be able to afford. I think they may be trying to get a few of the universities that use afs heavily to try and split the bill somehow :/

                          1. 2

                            I’ve had an account for a bit. It’s neat, but not something I feel I’d use regularly. You have a login password, which they’re aware of, and a second encryption password to decrypt your mail after logging in. Your private key for the encryption is stored on their servers. Prontonmail -> Protonmail emails are encrypted “end to end”, but it’s not clear to me what happens with your mail when a non-protonmail account sends you something… I think it just sits unencrypted until the next time you log in.

                            When you want to encrypt something for non-protonmail users, they get an (styled, with HTML/JS I think) email, asking them to click a link and enter a password that you gave them out of band to decrypt the message (all done in a browser).

                            Overall, it’s nice if you’re only emailing other people with protonmail accounts, but clunky if not. I’m not even sure that it provides a better UI than Enigmail/GPG + Thunderbird.

                            1. 2

                              Has anyone had much luck installing OpenBSD on an x220 with Intel HD graphics? Last time I tried (probably with OpenBSD 5.4) I couldn’t seem to get video acceleration working.

                              1. 3

                                5.5 snapshots ran fine on my x230 w/Intel HD gfx… though, I’m not sure if video acceleration was working. I was able to run Gnome3, so I think that counts?

                                1. 2

                                  Whoa, I installed 5.6 and it worked without a hitch. I also took the opportunity to switch over to cwm after using ratpoison for awhile… a reminder of how nice the defaults are.

                                  1. 2

                                    I have run accelerated graphics on my x220 for a while now. Check the perms on your /dev/drm* As it turns out I have “Intel HD Graphics 3000”

                                    1. 1

                                      Lots of luck, daily.

                                      Only think that i had issues with is my realtek wireless but i use a usb dongle to connect wirelessly until someone takes care of it.

                                    1. 7

                                      I wrote a quick commentary on some of the issues at hand.

                                      Crowdfunding needs some ways for non experts to be able to assess whether a certain project can deliver on its promises.(which ind.ie never could. Building your own smartphone with a handful of people?)

                                      1. 1

                                        I think it can be done. Not a smartphone that can compete with Samsung or Apple on (all of) performance, fit-and-finish, and size, but it can be done. Bunnie and Xobs are shipping their open-source Novena laptop as we speak. There’s only two of them, and they seem to have fingers in many pies to boot :)

                                        1. 2

                                          I don’t think this team could have. Bunnie and crew are technical people who have the knowledge, connections and know their limitations. These people didn’t seem to have a realistic goal. Maybe they can badge engineer a Chinese phone with some firmware, but even that I doubt.

                                          1. 2

                                            I agree that it can be done – isn’t this what Jolla did?

                                        1. 7

                                          I’ve heard a lot about ownCloud being poor in security and stability (as in: it struggles with large files) - are these still true?

                                          1. 3

                                            I was running owncloud up until a few months ago (on Debian-ish systems). The “official” package repositories had their GPG key expire, and I think it took months to update it. The Ubuntu/Canonical package stopped getting updated too, if I recall correctly. But, I suspect these issues wouldn’t be as much of a problem with the way the ports system works on the BSDs (and of course, you can always do a source installation on linux too).

                                            As for other security issues, I’d say that owncloud suffers from similar security issues as other well known PHP web apps (thinking Wordpress, Drupal, etc). But, being open source, we can hope those issues are caught quickly and updated.

                                            I don’t remember large file syncing to be too problematic. The client got a little CPU hungry every once in a while, but nothing terrible.

                                            In any case, I was only using the owncloud syncing, and wasn’t happy with the packaging issues, so I switched to Syncthing, and have been pretty happy with it. Clients and protocol are open, written in Go, overall pretty neat :)

                                            1. 2

                                              Ah, and to add, the only real thing I miss from owncloud was that they had mobile apps for iOS and Android, Syncthing is only on Android right now (but I think Go is available soon in iOS, so maybe it’ll show up there soon!).

                                          1. 7

                                            About to put the finishing touches on a Ceph Object Gateway deployment at work. We’ve currently got the storage cluster running and providing RBD volumes for our OpenStack cloud, but the object gateway will provide S3-like services on-site for our users.

                                            1. 10

                                              Peter’s actual blog post is up here (posted link is a torrentfreak article on the blog post): http://blog.brokep.com/2014/12/09/the-pirate-bay-down-forever/

                                              1. 9

                                                The official Puppet Labs packages will install it as a monstrous cylindrical tower.

                                                This made me lol >.<

                                                1. 2

                                                  This looks really cool. Having managed some larger infrastructures with puppet and its “eventually consistent” model, I’ve found that it doesn’t work all too well. Removing an IP from an nginx proxy like in the Consul template example video requires a human to edit our hiera data store, commit the change, and either run puppet by hand on the proxy, or wait ~30mins for the update. The realtime updates that can automatically remove the IP when the server fails just seem so awesome in comparison. :-)

                                                  1. 3

                                                    Working on setting up an instance of elastic search and so far it’s been going very very smoothly – of course the real challenge will go into seeing how well it can handle a lot of traffic. I’m using it on a table with about +800,000 rows in it that comes out to about 22mb as a file. I’ve tested in on about 1/20 of the data and searches load up onto the client within 110ms-10ms. Yay!

                                                    I’ll have to eventually do the same thing in sphinx, because the lead developer I’m working with only want to work with sphinx, and even though I have elasticsearch working already, he’d like to see it in sphinx to compare. So, this should be an interesting assignment…

                                                    If anybody has any thoughts on real time indexing, elastic search, and sphinx, please PM me and give me your thoughts!

                                                    1. 3

                                                      What kind of data/traffic are you planning on indexing with elasticsearch or sphinx? I’ve used elasticsearch with Logstash before, and never had too many problems (though, I didn’t have an insane amount of logs). I’ve heard that Dreamhost uses logstash/elasticsearch to index about 10,000 log events per second, so I imagine the elasticsearch part scales very well depending on how you set it up!

                                                      1. 1

                                                        I’ve heard differently. One of the senior devs that I work with told me he used Elastic Search for a much bigger project where he had it running on multiple nodes (3 or so) and he ran into huge syncing issues – which eventually lead to abandoning the project. The conclusion at my company came down to that elastic search isn’t stable enough for production use. I’m just experimenting with it – hence why I have to eventually reproduce everything with sphinx.

                                                        It might work out though since I really only need one node for a relatively small table. We’re using it to index the user names of the members of our site – which comes out to a little over ~800,000 (and on its way up) so that when a user is typing in a input box, it shows a drop down of a search of the members that match whatevers been typed into the input box currently.

                                                        1. 2

                                                          I’ve used ES in two previous projects, one small (a few hundred thousand docs) and one larger (single-digit millions). The former was done on a single node and eventually moved to heroku, the latter is more recent. It’s internally focused and only runs 3 nodes, but the response time is great and I haven’t run into too much syncing. That said, it’s not being used as a read/write store in either setup. Just a write-once-read-many setup where the search index is built in one big batch. For the smaller instance, this took about 5 seconds, the larger takes a few minutes, which is good enough. I suspect at very large scales you’d have to do incremental updates, but I'ven’t got there. Overall I’ve never had massive syncing issues running using that strategy on a few nodes, it also makes reasoning about where data lives canonically a lot easier.

                                                          Just some food for thought, good luck with your experiments.

                                                          1. 2

                                                            @aphyr’s excellent “Call Me Maybe” post on Elasticsearch is worth seeing. It explains some of the data loss behavior that your coworkers might have seen.

                                                            However, depending upon the kind of logs you’re searching, it may not matter. If this is your primary data store for the data you care the most about, that might be a bummer. If this is the journal for data where the source of truth is another data store, it might not matter so much. Also, if these are warnings and errors you’re logging from your application, it also might not matter that much.

                                                            1. 1

                                                              Thank’s for that. I’ll shall read that during my lunch break.

                                                              1. 1

                                                                That was such a heavy blog post… those barbie gifs don’t do much justice to the extent and depth of the content in there. So comcast paid him to play around with elastic search, huh…