1. 4

    Make the weekly! We do standups every Monday and have a slack channel where you post what you’re working on that day.

    1. 1

      Well yes there are limitations… I think you can work around most of them though. So maybe there was too much hype. But AI is here to stay. It will change everything because it allows more automation. It’s just really far away from a general AI, or sentient AI. Pattern recognition on steroids would be a better name, though not so sexy as AI.

      1. 1

        This looks like a nice upgrade from autopep8

        1. 1

          Featuring lobste.rs in the onboarding flow: https://github.com/GetStream/Winds/blob/master/api/src/workers/featured.json#L474

          Interesting to learn more about how you stay up to date on news/tech. Are you still using RSS, relying on Twitter, mailing lists, Reddit etc…

          1. 1

            What does your backend setup look like for something like this? Would it be possible for you to allow self hosting in the future?

            1. 2

              The backend API and frontend are both open source: https://github.com/getstream/winds We rely on Algolia, Stream and Mercury though which are closed source. Now that doesn’t stop you from running your own backend or changing the app’s functionality as you see fit.

            1. 3

              There are many companies offering to help with GDPR for 6 figure amounts. Cost of compliance is in the millions for many larger companies. (this author clearly doesn’t understand the true cost of things)

              So far there are no real privacy benefits for me as a user. I don’t care about people tracking my IPs personally, or running analytics, retargeting, or doing split testing. I care about people losing access to my passwords, social, credit cards, messages, pictures, location data etc. I haven’t seen much improvement in that area. End result so far seems to be more checkboxes and the ability to delete my user account. #awesome

              It’s too early to tell if anything good will come out of GDPR. Fingers crossed though, there are real privacy issues to solve and I hope it helps with that.

              1. 5

                Why ask for our work email to download it? It comes across as very creepy and also unfortunate for those of us who don’t have one…

                1. 4

                  well… it used to be a beta registration page. which we copied from another landing page that required your work email. it doesn’t make sense at all and I’ll remove it on the next deploy.

                  which platform are you on?

                  1. 1

                    Ah totally fair :) Mac at the moment

                    1. 2

                      I’m still fine-tuning the build for Mac and the app store. You can try it out here (but it won’t automatically update) https://s3.amazonaws.com/winds-2.0-releases/releases/Winds-2.0.173.pkg

                      1. 1

                        Thanks and best of luck!

                1. 2

                  I would say JS already won. I prefer Python & Go. Still think Django or Rails are miles ahead in terms of productivity. However the vast majority of new projects are choosing for Node. We actually use Node for all our example/marketing projects since it’s just so much more popular than other languages. One interesting development is that Node adopted most of Python’s features over the past years, it really improved as a language. I still don’t like the async callback approach to handling concurrency, but other than that its a pretty decent language.

                  1. 4

                    I don’t know, depending on your company, country and position you may feel this differently. In my few working experiences, people would run away from Javascript. Backend people moving to golang majoritarily and frontend people going from javascript and moving to Typescript, Elm or Reason. I think it totally depends of where, who and what you’re working on…

                    1. 2

                      Do people use Go for enteprise-y CRUD apps? I see a lot of Go for services and things that require eating through a bunch of data but I don’t hear about its use in other domains

                      1. 2

                        We use golang in this capacity. Backend services that don’t need tons of front end tooling are really nice to write in golang.

                        1. 1

                          Have you been basically hand-rolling most of the functionality (thinking in particular about ORMs and outputting HTML for the client)

                          To be honest when reading Go code it tends to look very “nice C”-y, but that feels like it might lead to frustration when dealing with a bunch of strings to concatenate.

                          1. 1

                            haha, why yes we have: https://github.com/blend/go-sdk

                            the golang stdlib gets you most of the way there, that sdk is really just a web helper, a logging / eventing helper, and a lite orm with a bunch of other random stuff thrown in for services that needed it

                        2. 1

                          Yes we did. As API serving JSON but also serving templates HTML. It’s quite nice but to be honest we didn’t grow it too big so we didn’t have much trouble maintaining it.

                    1. 1

                      Well yes a lot of their issues are caused by having APIs that are too open. To be fair, back in those days, the tech ecosystem was definitely pushing for this openness. It was considered a good thing. Now, not so much..

                      1. 1

                        In our buzzwords-driven field?

                        Probably people considered API access “a good thing” just because “Facebook/Google is doing this too!”

                        But the problem was not the technology back then, just like AI is not the solution right now.

                        It’s the business model.

                        I remember a younger Zuckerberg explaining the world how privacy had no value for modern people.

                        He meant it!

                        1. 1

                          back in those days, the tech ecosystem was definitely pushing for this openness.

                          I would hardly call 2015 “those days”.

                          1. 1

                            Back in my day…

                        1. 1

                          This is really cool :) Fun read, thanks for sharing.

                          1. 3

                            It’s impressive how much money they raised. Wonder how well their monetization is working for them.

                            1. 19

                              sighs

                              Replacing a corporate data aggregator with a distributed one doesn’t actually reduce the amount of data gathered.

                              If you don’t want your information online and searchable don’t put it online.

                              It doesn’t matter if it’s a friendly mastadon instead of a Harvard dudebro–sharing data means your data is shared. Staaaaaahp.

                              EDIT: Mastadon also has some interesting history.

                              1. 34

                                If you don’t want your information online and searchable don’t put it online.

                                This is not a panacea. Facebook has my phone number because other people chose to upload their contacts. Google has incredibly personal conversations because other people chose them for email. Equifax has my credit history because nearly every banking institution reports to them. Nielsen-Catalina Solutions knows my shopping preferences because retailers secretly sell it to them.

                                If you don’t want your information online and searchable, get data protection laws.

                                1. 3

                                  Laws help, but we also have to take responsibility for not sharing our data (or the data of our friends) online.

                                  1. 1

                                    Unfortunately most users don’t know or don’t care that Facebook uploads their contacts.

                                2. [Comment removed by author]

                                  1. 6

                                    Please elaborate. I thought it was an interesting look into experience of having vastly different cultures using the same messaging fabric, and the issues that that gives rise to.

                                    1. 2

                                      I don’t think it’s garbage. I think it could have been better written, but as you point out the culture clash thing is an interesting phenomena.

                                      I also don’t think said history would have any bearing on which social media platform you choose for most people.

                                    2. 2

                                      That article is absolute, complete garbage.

                                      Do you see it as garbage because of an abundance of factual inaccuracies, or something else?

                                      The reason I ask is that clearly there’s an absolutist free-speech position being promoted, but certainly all the stuff about Japanese and Spanish speaking Mastodon activity correlates well with what I saw at the time. I don’t know anything about people getting upset about Eugen being paid though, or any of the behind the scenes stuff.

                                    3. 5

                                      Replacing a corporate data aggregator with a distributed one doesn’t actually reduce the amount of data gathered.

                                      It does if the data you share is subject to aggregator influence. And it is, since the aggregator controls the platform and its defaults.

                                      Facebook went through a period where everytime I checked my privacy settings I found something open that I didn’t want to be open. The years of the Cambridge Analytica scrape line up pretty well with that phenomenon. Facebook used to be hugely incented to make as much of your data public to the world (search engines and, it turns out, CA) as possible. Mastodon has no such incentives.

                                      Yes, if I share something with someone I share it with them. But I’d like to not share it with everyone else.

                                      1. 1

                                        It does if the data you share is subject to aggregator influence

                                        I’m not quite sure what this means, do you mind elaborating?

                                        1. 5

                                          I thought I did in the rest of my comment? Basically I’d enter some data in my profile with some understanding of what was visible to whom. Then I’d come back a month or three later, and somehow stuff I intended to be visible only to friends would somehow be visible to some new vector (apps) or API. Facebook’s privacy settings sprawled out of control for a couple of years. Here’s some links I was able to dig up in a quick search:

                                          http://mattmckeon.com/facebook-privacy

                                          https://www.eff.org/deeplinks/2009/12/facebooks-new-privacy-changes-good-bad-and-ugly

                                          https://www.eff.org/deeplinks/2010/04/facebook-timeline

                                          https://www.washingtonpost.com/news/the-switch/wp/2014/04/08/your-facebook-privacy-settings-are-about-to-change-again

                                      2. 3

                                        I agree with this sentiment but I think all the bruhaha is currently about something entirely different. When you use an account on Mastodon, your toots are federated across the global timeline. That, along with an email address that stays local to the server you signed up on, and maybe some HTTPS traffic logs on your server, is the sum total of the information you are exposing via Mastdon until you choose to add more.

                                        This is, from where I stand at least, a vastly different kettle of fish than Facebook.

                                        1. 2

                                          I agree. To some extend the distributed nature even makes it harder to remove data you don’t want online anymore.

                                          1. 3

                                            Removing data is already impossible in the information-theoretical sense. You just get lucky a lot of the time.

                                            To address this particular issue, IPFS has blacklists that track DMCA notices and abusive content. They’re opted in to by consumers.

                                            1. 2

                                              On the other hand the data is also distributed across many instances as opposed to being owned by a single entity. There’s also the fact that Mastodon doesn’t try to track your personal identity, and the interactions can be completely anonymous. Meanwhile, the whole purpose of a site like Facebook is to build an intimate profile of you and your friends.

                                              1. 1

                                                Depends, some instances have ElasticSearch enabled, ostensibly to enable full-text search, but ES can be used for more insidious ““big data” purposes, to profile users with. Tools like Kibana from the ES people make such tasks trivial compared to writing tedious queries by hand. And due to the nature of federation, if someone from that instance follows you, they have your toots, which the admin can use for said purposes.

                                          1. 35

                                            I’ll bite.

                                            General industry trends
                                            • (5 years) Ready VC will dry up, advertising revenue will bottomout, and companies will have to tighten their belts, disgorging legions of middlingly-skilled developers onto the market–salaries will plummet.
                                            • (10 years) There will be a loud and messy legal discrimination case ruling in favor of protecting political beliefs and out-of-work activities (probably defending some skinhead). This will accelerate an avalanche of HR drama. People not from the American coasts will continue business as usual.
                                            • (10 years) There will be at least two major unions for software engineers with proper collective bargaining.
                                            • (10 years) Increasingly, we’ll see more “coop” teams. The average size will be about half of what it is today, organized around smaller and more cohesive business ideas. These teams will have equal ownership in the profits of their projects.
                                            Education
                                            • (5 years) All schools will have some form of programming taught. Most will be garbage.
                                            • (10 years) Workforce starts getting hit with students who grew up on touchscreens and walled gardens. They are worse at programming than the folks that came before them. They are also more pleasant to work with, when they’re not looking at their phones.
                                            • (10 years) Some schools will ban social media and communications devices to promote classroom focus.
                                            • (15 years) There will be a serious retrospective analysis in an academic journal pointing out that web development was almost deliberately constructed to make teaching it as a craft as hard as possible.
                                            Networking
                                            • (5 years) Mesh networks still don’t matter. :(
                                            • (10 years) Mesh networks matter, but are a great way to get in trouble with the government.
                                            • (10 years) IPv6 still isn’t rolled out properly.
                                            • (15 years) It is impossible to host your own server on the “public” internet unless you’re a business.
                                            Devops
                                            • (5 years) Security, cost, and regulatory concerns are going to move people back towards running their own hardware.
                                            • (10 years) Containers will be stuck in Big Enterprise, and everybody else will realize they were a mistake made to compensate for unskilled developers.
                                            • (15 years) There will still be work available for legacy Rails applications.
                                            Hardware
                                            • (5 years) Alternative battery and PCB techniques allow for more flexible electronics. This initially only shows up in toys, later spreads to fashion. Limited use otherwise.
                                            • (5 years) VR fails to revitalize the wounded videocard market. Videocard manufacturers are on permanent decline due to pathologies of selling to the cryptobutts folks at expense of building reliable customer base. Gamers have decided graphics are Good Enough, and don’t pay for new gear.
                                            • (10 years) No significant changes in core count or clock speed will be practical, focus will be shifted instead to power consumption, heat dissipation, and DRM. Chipmakers slash R&D budgets in favor of legal team sizes, since that’s what actually ensures income.

                                            ~

                                            I’ve got other fun ones, but that’s a good start I think.

                                            1. 7

                                              (5 years) Security, cost, and regulatory concerns are going to move people back towards running their own hardware.

                                              As of today, public cloud is actually solving several (and way more than people running their own hardware) of these issues.

                                              (10 years) Containers will be stuck in Big Enterprise, and everybody else will realize they were a mistake made to compensate for unskilled developers.

                                              Containers are actually solving some real problems, several of them already were independently solved, but containers bring a more cohesive solution.

                                              1. 1

                                                Containers are actually solving some real problems, several of them already were independently solved, but containers bring a more cohesive solution.

                                                I am interested, could you elaborate?

                                                1. 1

                                                  The two main ones that I often mention in favor of containers (trying to stay concise):

                                                  • Isolation: We previously had VMs on a virtualization level but they’re heavy, potentially slow to boot and obscure (try to launch xen and manage vms your pet server), and jail/chroot are way harder to setup and specific to each of your application and do not allow you to restrict resources (to my knowledge).
                                                  • Standard interface: Very useful for orchestration as an example, several tool existed to deploy applications with an orchestrator, but it was mostly executables and suffered from the lack of isolation. Statically compiling solved some of theses issues, but not every application can be.

                                                  Containers are a solution to some problems but not the solution to everything. I just think that wishing they weren’t there, probably means the interlocutor didn’t understand the benefits of it.

                                                  1. 2

                                                    I just think that wishing they weren’t there, probably means the interlocutor didn’t understand the benefits of it.

                                                    I’ve been using FreeBSD jails since 2000, and Solaris zones since Solaris 10, circa 2005. I’ve been writing alternative front-ends for containers in Linux. I think I understand containers and their benefits pretty well.

                                                    That doesn’t mean I don’t think docker, and kubernetes, and all the “modern” stuff are not a steaming pile, both the idea and especially the implementation.

                                                    There is nothing wrong with container technology, containers are great. But there is something fundamentally wrong with the way software is deployed today, using containers.

                                                    1. 1

                                                      But there is something fundamentally wrong with the way software is deployed today, using containers.

                                                      Can you elaborate? Do you have resources to share on that? I feel a comment on Lobsters might a be a bit light to explain such a statement.

                                                    2. 1

                                                      You can actually set resource isolation on various levels; classic Unix quotas, priorities (“nice” in sh) and setrusage() (“ulimit” in sh); Linux cgroups etc. (which is what Docker uses, IIUC); and/or more-specific solutions such as java -Xmx […].

                                                      1. 2

                                                        So you have to use X different tools and syntax to, set the CPU/RAM/IO/… limits, and why using cgroups when you can have cgroups + other features using containers? I mean, your answer is correct but in reality, it’s deeply annoying to work with these at large scale.

                                                        1. 4

                                                          Eh, I’m a pretty decent old-school sysadmin, and Docker isn’t what I’d consider stable. (Or supported on OpenBSD.) I think this is more of a choose-your-own-pain case.

                                                          1. 3

                                                            I really feel this debate is exactly like debates about programming languages. It all depends of your use-cases and experience with each technologies!

                                                            1. 2

                                                              I’ll second that. We use Docker for some internal stuff and it’s not very stable in my experience.

                                                              1. 1

                                                                If you have <10 applications to run for decades, don’t use Docker. If you have +100 applications to launch and update regularly, or at scale, you often don’t care if 1 or 2 containers die sometimes. You just restart them and it’s almost expected that you won’t reach 100% stability.

                                                                1. 1

                                                                  I’m not sure I buy that.

                                                                  Out testing infrastructure uses docker containers. I don’t think we’re doing anything unusual, but we still run into problems once or twice a week that require somebody to “sudo killall docker” because it’s completely hung up and unresponsive.

                                                                  1. 1

                                                                    We run at $job thousands of container everyday and it’s very uncommon to have containers crashing because of Docker.

                                                      2. 1

                                                        Easier local development is a big one - developers being able to quickly bring up a full stack of services on their machines. In a world of many services this can be really valuable - you don’t want to be mocking out interfaces if you can avoid it, and better still is calling out to the same code that’s going to be running in production. Another is the fact that the container that’s built by your build system after your tests pass is exactly what runs in production.

                                                    3. 7

                                                      (5 years) VR fails to revitalize the wounded videocard market. Videocard manufacturers are on permanent decline due to pathologies of selling to the cryptobutts folks at expense of building reliable customer base. Gamers have decided graphics are Good Enough, and don’t pay for new gear.

                                                      While I might accept that VR may fail, I don’t think video card companies are reliant on VR succeeding. They have autonomous cars and machine learning to look forward to.

                                                      1. 2

                                                        (10 years) No significant changes in core count or clock speed will be practical, focus will be shifted instead to power consumption, heat dissipation, and DRM. Chipmakers slash R&D budgets in favor of legal team sizes, since that’s what actually ensures income.

                                                        This trend also supports a shift away from scripting languages towards Rust, Go, etc. A focus on hardware extensions (eg deep learning hardware) goes with it.

                                                        1. 1

                                                          (10 years) Containers will be stuck in Big Enterprise, and everybody else will realize they were a mistake made to compensate for unskilled developers.

                                                          One can dream!

                                                          1. 2

                                                            Would you (or anyone) be able to help me understand this point please? My current job uses containers heavily, and previously I’ve used Solaris Zones and FreeBSD jails. What I see is that developers are able to very closely emulate the deployment environment in development, and don’t have to do “cross platform” tricks just to get a desktop that isn’t running their server OS. I see that particular “skill” as unnecessary unless the software being cross-platform is truly a business goal.

                                                            1. 1

                                                              I think Jessie Frazelle perfectly answer to this concern here: https://blog.jessfraz.com/post/containers-zones-jails-vms/

                                                              P.S.: I have the same question to people that are against containers…

                                                          2. 1

                                                            (5 years) Mesh networks still don’t matter. :( (10 years) Mesh networks matter, but are a great way to get in trouble with the government.

                                                            Serious attempts at mesh networks basically don’t exist since the 200#s when everyone discovered it’s way easier to deploy an overlay net on top of Comcast instead of making mid-distance hops with RONJA/etc.

                                                            It would be so cool to build a hybrid USPS/UPS/Fedex batch + local realtime link powered national scale network capable of, say, 100mB per user per day, with ~ 3 day max latency. All attempts I’ve found are either very small scale, or just boil down to sending encrypted packets over Comcast.

                                                            1. 1

                                                              Everyone’s definition of mesh different, but today there are many serious mesh networks, the main ones being Freifunk and Guifi

                                                            2. 1

                                                              (10 years) There will be at least two major unions for software engineers with proper collective bargaining.

                                                              What leads you to this conclusion? From what I hear, it’s rather the opposite trend, not only in the software industry…

                                                              (5 years) All schools will have some form of programming taught. Most will be garbage.

                                                              …especially if this is taken into account, I’d argue.

                                                              (10 years) Some schools will ban social media and communications devices to promote classroom focus.

                                                              Aren’t these already banned from schools? Or are you talking about general bans?

                                                              1. 1

                                                                I like the container one, I also don’t see the point

                                                                1. 1

                                                                  It’s really easy to see what state a container is in because you can read a 200 line text file and see that it’s just alpine linux with X Y Z installed and this config changed. On a VM it’s next to impossible to see what has been changed since it was installed.

                                                                  1. 3

                                                                    ate a container is in because you can read a 200 line text file and see that it’s just alpine linux with X Y Z in

                                                                    I just check the puppet manifest

                                                                    1. 2

                                                                      It’s still possible to change other things outside of that config. With a container having almost no persistent memory if you change something outside of the dockerfile it will be blown away soon.

                                                                  2. 1

                                                                    Containers wont be needed because unikernels.

                                                                  3. 1

                                                                    All schools will have some form of programming taught. Most will be garbage.

                                                                    and will therefore be highly desirable hires to full stack shops.

                                                                    1. 1

                                                                      I would add the bottom falling out of the PC market, making PCs more expensive as gamers and enterprise, the entire reason why it still maintains economies of scale, just don’t buy new HW anymore.

                                                                      1. 1

                                                                        I used to always buy PCs, but indeed the last 5 years I haven’t used a desktop PC.

                                                                        1. 1

                                                                          If it does happen, It’ll probably affect laptops as well, but desktops especially.

                                                                      2. 1

                                                                        (5 years) All schools will have some form of programming taught. Most will be garbage.

                                                                        My prediction: Whether the programming language is garbage or not, provided some reasonable amount of time is spent on these courses we will see a general improvement in the logical thinking and deductive reasoning skills of those students.

                                                                        (at least, I hope so)

                                                                      1. 1

                                                                        Cloudformation, Cloud-Init, Puppet, Boto and Fabric. Works like a charm, but none of these tools are perfect.

                                                                        1. 1

                                                                          Thanks for posting!

                                                                          I want to remark on story submission guideline: It says “Commentary or opinion should be reserved for a comment, so that it can be voted on separately from the story”. I found both Instagram’s and Stream’s story worth upvoting, but I can’t upvote both as-is. Maybe next time!

                                                                          1. 1

                                                                            Thanks for pointing that out. RocksDB is quite an impressive building block.

                                                                          1. 6

                                                                            Well.. This is just a really hard problem and something many large companies struggle with. The startup or freelance route he’s going down is in many ways much harder though.

                                                                            1. 4

                                                                              Uh yeah, he’s gone from something with a little risk of BS and setbacks to something with a large amount of both. Maybe Id buy it if he just said consulting with his Google skills and creds. Instead, he’s basically bootstrapping with stuff he doesnt have a business model for yet. He’ll also have to do similar feature focus to appeal to most software markets.

                                                                              He might have been better off just picking a different company with better incentives or consulting in his skill areas. I sent him a link to Barnacl.es anyway along with a “Good luck!” Maybe he’ll be in lucky percentile [again after getting hired by Google].

                                                                            1. 0

                                                                              I really like Go, for these reasons: https://getstream.io/blog/switched-python-go/

                                                                              I don’t think any other language comes close for this use case:

                                                                              • the task i’m working on is performance intensive
                                                                              • it doesn’t require me to use CGO (go is not great in this area)
                                                                              • regex and json parsing performance isn’t super important (java & c++ still do this better)
                                                                              • development time is important (Java and C++ have their advantages for some things, but it takes more time to build things)

                                                                              Also, I like how every release of Go improves something i care about. For instance 1.10 substantially reduced the time it takes for code to compile. 1.9 shaved a few milliseconds of our ranked feeds. It’s awesome that stuff you don’t work on just improves as time goes by.

                                                                              1. 2

                                                                                Totally agree with the author. I was also using Python for too many things. Nowadays it’s Go for most things, Python for simple scripts, Node for some other stuff (running a test suite against websockets is much easier in Go than most other languages, also scraping is pretty awesome in Node)

                                                                                1. 2

                                                                                  r. I was also using Python for too many things. Nowadays it’s Go for most things, Python for simple scripts, Node for some other stuff (running a test suite against websockets is much easier in Go than most other languages, also scraping is pretty awesome in Node)

                                                                                  Python is more ergonomic for some CRUD stuff when you can just use django, other than that I prefer Go.

                                                                                1. 2

                                                                                  happy to implement a little notification feed with Stream if people think it’s useful. pretty easy to aggregate it and plug in realtime WS based updates.

                                                                                  1. 2

                                                                                    Also, anyone else here at lobsters using RocksDB?