1. 2

    $home: I’ve been working through the advent of code challenges live on stream (recordings) so I plan to continue doing that this week or at least for however long I make it. Waking up at 6:30 takes quite a toll on me as I can’t seem to fall asleep early enough to get a good eight hours (which seems to be the minimum I require per night) so I’ve mostly been going on 5 and a half to 6 hours’ sleep per night so far. There are a couple PRs to Dramatiq that I’m going to have to merge and one to Molten that I’m going to have to pick up b/c the original author can’t finish it.

    1. 1

      How many viewers? Any revenue from streaming?

    1. 41

      Wow, that’s pretty terrible.

      On the other hand, I can’t help but to feel sorry about Dominic, we all make mistakes, this public shaming is pretty violent.

      I guess we should sometimes take some time off to read the license before using a library:

      THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

      (F)OSS is not a consumer good.

      1. 11

        I agree that shaming people is toxic and unproductive. No one wants to be shamed and no one is perfect.

        But I see another dimension to the negative responses Dominic has received. Non-hierarchical, self-governing communities like open source software are organized by social norms. Social norms work through peer pressure - community members conform to the norms of the community not because they are compelled to by law but because it would cost them standing in the community not to. This isn’t inherently good. Some norms are toxic and self-policing via peer pressure can lead to shaming. What I see in some of the critical comments addressed to Dominic is an attempt to establish a clear social norm about what to do when you are ready to abandon a package. The norm is desirable because it increases the general level of trust. Even if the landscape is generally untrustworthy, you can have some confidence that people aren’t handing their packages off to strangers because it’s the norm not to do that. The desire for some norm here, whatever it is in the end, is reasonable.

        Ending the discussion with “don’t worry about it Dominic, everyone makes mistakes, and anyways you’re not liable for it” signals to everyone that they’re not responsible for the consequences of what they do. In a strictly legal sense, that might be true. Even then, I’m skeptical that the warranty clause would cover negligence in the distribution of the software rather than the software itself. But in either case, don’t we want a community where people do feel responsible for the actions they take and are open to receiving feedback when an action they’ve taken has a bad result? This dialogue can occur without shaming, without targeting anyone personally, and can be part of the same give-and-take process that produces the software itself.

        1. 6

          Blaming people in any security issue is toxic, no matter what happens. In any organization with paid people where you should expect better, the most important rule of a post-mortem is to remain blameless. It doesn’t get anyone anywhere and doesn’t get remotely close to actual root cause. Instead of asking about why Dominic gave away a critical package, people should be asking why some random maintainer were able to give away a critical package.

          Ending the discussion with “don’t worry about it Dominic, everyone makes mistakes, and anyways you’re not liable for it” signals to everyone that they’re not responsible for the consequences of what they do.

          By putting blame on Dominic, people are not taking responsibilities. The main issue is that many core libraries in the JavaScript ecosystems still depends on external, single-file, non-core, likely unmaintained library. People who should take responsabilities are the ones who chose to add a weak single point of failure by depending on event-stream.

          1. 2

            It depends what you mean by blame. If you mean assigning moral responsibility, especially as a pretext for shaming them, then I agree it’s toxic. I think I was clear that I agree this shouldn’t happen. But if blame means asserting a causal relationship between Dominic’s actions and this result, it’s hard to argue that there isn’t such a relationship. The attack was only possible because Dominic transferred the package. This doesn’t mean he’s a bad person or that he should be “in trouble” or that anything negative should happen to him as a consequence. A healthy social norm would be to avoid transferring packages to un-credentialed strangers when you’re ready to abandon the package because we’ve seen this opens an attack vector. Then what’s happened here is instructive and everyone benefits from the experience. And yes, ideally these dilemmas are prohibited by the system. Until that is the case, it helps to have norms around the best way to act.

            1. 1

              I understand you don’t condone the attacks and shaming going around. However I would argue that even if you agree that the blaming is toxic, that building some social norm around it is better than nothing, I believe that even hinting that it was somehow Dominic’s fault is a net negative.

              The attack was only possible because Dominic transferred the package.

              This is exactly what I’m condoning. By looking at individual and their action you scope the issue at that level. The attack was taking over a dependancy. It is possible to do so in so many way, especially for packages such as Dominic’s. This time it was a case of social engineering, next time it might as well be a case of credential hijacking, phishing or maintainer going rogue.

              A healthy social norm would be to avoid transferring packages to un-credentialed strangers when you’re ready to abandon the package because we’ve seen this opens an attack vector.

              I would say pushing this rethoric is actually unhealty and only lead people to rely on those social norm and use it as an excuse to disown their accountability. It would be much healthier to set expectation right and learn proper risk assessment about dependancies management.

              Then what’s happened here is instructive and everyone benefits from the experience. And yes, ideally these dilemmas are prohibited by the system. Until that is the case, it helps to have norms around the best way to act.

              The same issue have come up so many time in the past few years, especially in the NPM ecosystem, it should be well past the “learn from the experience” and I believe it’s time the relevant actors actually move toward a solution.

        2. 17

          I’ve done a similar thing before. After leaving the Elm community, I offered to transfer most of my repos over to the elm-community organisation. They accepted the most popular ones, but not elm-ast (and maybe one or two others). A few months later I received an e-mail from @wende asking if he could take over so I took a look at his profile and stuff he’s done in the past and happily gave him commit access thinking users would continue getting updates and improvements without any hassle. Now, @wende turns out to be a great guy and I’m pretty sure he hasn’t backdoored anyone using elm-ast, but I find it hilarious that people somehow think that maintainers should be responsible for vetting who they hand over control of their projects to or that they could even do a good job of it OR that it would even make any sort of a difference. Instead of trusting one random dude on the internet (me) you’re now trusting another.

          Don’t implicitly trust random people on the internet and run their code. Vet the code you run and keep your dependency tree small.

          1. 25

            Vet the code you run

            Or trust well-known, security-oriented distributions.

            keep your dependency tree small

            Yes, and stay away from environment, frameworks, languages that force dependency fragmentation on you.

            1. 4

              Or trust well-known, security-oriented distributions.

              That too! :D

              1. 3

                and stay away from […] frameworks

                I wouldn’t say that as absolutely for the web. I suspect that things would likely go a lot more haywire if people started handling raw HTTP in Python or Ruby or what have you. There’s a lot of stuff going on under the hood such as content security policies, CSRF protection and the like. If you’re not actively, consciously aware of all of that, a web framework will probably still end up providing a net security benefit.

                1. 5

                  Please don’t quote words without context:

                  […] that force dependency fragmentation on you

                  Frameworks and libraries with few dependencies and a good security track record are not the problem. (If anything, they are beneficial)

                  1. 2

                    I interpreted “Yes, and stay away from environment, frameworks, languages that force dependency fragmentation on you.” as (my misunderstandings in brackets) “Yes, and stay away from [(a) integrated development] environments, [(b)] frameworks, [(c)] languages that force dependency fragmentation on you.” with a and b being separate from the “that” in c.

                    I apologize for the misunderstanding caused.

                2. 2

                  Isn’t it the case that reputable, security-focused distributions acquire such status and the continuity thereof by performing extensive vetting of maintainers?

                  The responsible alternative being abandoning the project and letting the community fork it if they want to.

                  1. 1

                    Or trust well-known, security-oriented distributions.

                    Then how do You deal with things like that: “The reason the login form is delivered as web content is to increase development speed and agility” ?

                    1. 2

                      As a distribution? Open a bug upstream, offer a patch, and sometimes patch the packaged version.

                      1. 1

                        That’s a good idea in general but sometimes the bug is introduced downstream.

                3. 9

                  Most proprietary software also comes with pretty much the same warranty disclaimer. For example, see section 7c of the macOS EULA:

                  https://images.apple.com/legal/sla/docs/macosx107.pdf

                  I mean, have we held accountable Apple or Google or Microsoft or Facebook in any substantial ways for their security flaws?

                  1. 4

                    In many other products accountability is enforced by law and it overrides any EULA. And that is tied to profit in the broad sense: sales or having access to valuable customer data & so on.

                    Software companies got away with zero responsibility and this only encourages bad software.

                    1. 1

                      And how have we enforced that by law for those companies, regardless of what those EULAs have said? When macOS allowed anyone to log in as root, what were the legal consequences it faced?

                      1. 3

                        other products

                        e.g. selling cars without safety belts, electrical appliances without grounding…

                  2. 2

                    It is a security disaster given how easy it is for js stuff to hijack cookies and sessions.

                    1. 1

                      It really isn’t if a well thought out CORS policy is defined.

                  1. 2

                    $home: I’ve started working on (and streamingpast recordings here) a geolocation library for Racket based on the free DBs that MaxMind provides. I’m going to work on documenting and releasing it towards the end of the week. Streaming definitely slows me down, but I’ve found it really enjoyable so far and I like the sense of improvement that I get after each stream (re-watching my first and my latest stream there’s quite a lot that has improved IMO (and a lot more room to improve still)).

                    1. 7

                      Battlestation

                      “Desktop”/Emacs - my setup’s been one-full-screen-app-per-workspace for years now and I find it helps me focus. I have bindings to toggle between Emacs, iTerm and Chrome:

                      • C-c M-a from Emacs switches to iTerm,
                      • C-c M-a from iTerm switches to Emacs, and
                      • C-c M-c from either Emacs or iTerm switches to Chrome.
                      1. 3

                        I agree with most of this.

                        The lack of docstrings in particular is weird given that overall there’s a big emphasis on documentation. I disagree that the error messages are difficult to read; this kind of thing is pretty impressive and I wish I had it in other languages: http://p.hagelb.org/drracket-error.png The section on consistency is kind of weird; writing new code is quite consistent, but the language has been around a looooong time, and parts of the language haven’t aged that well. In particular the class system is kinda clunky, and there are older libraries that assume mutable data structures. But unless you’re doing a GUI program that is pretty easy to avoid.

                        1. 2

                          I disagree that the error messages are difficult to read; this kind of thing is pretty impressive and I wish I had it in other languages: http://p.hagelb.org/drracket-error.png

                          That is cool! Not being a Dr. Racket user, though, I’m missing out on that.

                          The section on consistency is kind of weird; writing new code is quite consistent, but the language has been around a looooong time, and parts of the language haven’t aged that well. In particular the class system is kinda clunky, and there are older libraries that assume mutable data structures. But unless you’re doing a GUI program that is pretty easy to avoid.

                          It’s possible (likely, even) I just haven’t touched the clunky stuff yet. I’ve used a couple of the functions in the GUI package, but nothing major so far and I haven’t touched the class system at all apart from reading about it in the user guide.

                        1. 4

                          This is an interesting idea, and has been pursued before.

                          For example, verbal expressions (in particular, since this is python: PythonVerbalExpressions, which does support Python 3).

                          What is the benefit of cursive_re over verbal expressions (which, though perhaps not standardized in a formal document, already has many implementations among many languages)? More than that, I do have to wonder where the line is between verbosity and terseness where gaining clarity levels off.

                          Regex are notoriously difficult for new-comers to grok (though I have several arguments about why that might be unrelated to their construction and lexicon); but there are always diminishing returns on clarity when you increase verbosity.

                          OP, where is the line for you? What led you to choose particular names over other (perhaps shorter) ones? For example, why name . anything() instead of any_single_character()?

                          This is not meant as an attack on this project, or any other that aims to make a terse language more readable. Rather, I am interested in the wider implication. Some programmers (e.g., APLers and Haskellers) clearly prefer greater information density; others (e.g., Java) tend to prefer the maximal verbosity so there can be no misinterpretation. I do not believe we have found a reasonable resolution to this dichotomy yet.

                          To inject my own thoughts now: I tend to err toward concision. Higher information density requires higher overhead for both reading and writing, but “sends less bits over the wire” (metaphorically and mechanically speaking), and results in fewer lines of code (and thus hopefully, fewer places where bugs can arise).

                          1. 3

                            For example, verbal expressions (in particular, since this is python: PythonVerbalExpressions, which does support Python 3).

                            What is the benefit of cursive_re over verbal expressions (which, though perhaps not standardized in a formal document, already has many implementations among many languages)? More than that, I do have to wonder where the line is between verbosity and terseness where gaining clarity levels off.

                            I had been vaguely aware of verbal expressions but never dove in because I don’t feel like mutability is a good fit for this sort of solution (because it doesn’t compose). cursive_re is closer in spirit to something like Emacs’ rx module.

                            OP, where is the line for you? What led you to choose particular names over other (perhaps shorter) ones? For example, why name . anything() instead of any_single_character()?

                            On the one hand I wanted to avoid clashing with names of builtin functions, so I couldn’t go with any for anything and on the other it’s just my personal preference for things to have meaningful names. My benchmark is: with minimal context, can I get a sense of what this thing does just by reading its name? I do the same regardless of whether I’m working in a terse language like Haskell or something like Java or Python in this case.

                            To inject my own thoughts now: I tend to err toward concision. Higher information density requires higher overhead for both reading and writing, but “sends less bits over the wire” (metaphorically and mechanically speaking), and results in fewer lines of code (and thus hopefully, fewer places where bugs can arise).

                            Code is read more often than it is written and maximally terse code tends to require maximum effort to read and change down the line. That’s not to say that ergonomics don’t matter (I’ve often argued that they do!), only that, oftentimes, being more verbose is a better strategy in the long run.

                            1. 2

                              Thank you for the thoughtful and clarifying response.

                              I had been vaguely aware of verbal expressions but never dove in because I don’t feel like mutability is a good fit for this sort of solution (because it doesn’t compose). cursive_re is closer in spirit to something like Emacs’ rx module.

                              Fair enough! I am not aware of Emacs’s rx module, nor am I familiar enough with PythonVerbalExpressions to speak on its use of mutability or its lack of composability.

                              Code is read more often than it is written and maximally terse code tends to require maximum effort to read and change down the line.

                              In that case, do you think Literate Programming in concert with concision is a reasonable solution, or would you prefer as you seem to have aimed for with cursive_re “self-documenting code”?

                              1. 3

                                In that case, do you think Literate Programming in concert with concision is a reasonable solution, or would you prefer as you seem to have aimed for with cursive_re “self-documenting code”?

                                My preference would be for self-documenting code. I think of comments as a means of documenting the whys of an implementation, which is similar to how literate programming is generally used, but most code doesn’t need that level of “documentation” (I’m using quotes here because I think this is different from reference documentation or guides (i.e. “docs”)). Additionally, code interacts with code: when you interact with a library written in a literate programming style, you most likely won’t have the literate part readily available to you to inspect.

                          1. 5

                            regexen may look like line noise, but I like to think of them akin to mathematical formulas. Their brevity in their own specialised DSL makes it much more easy to grasp at a glance what a regular expression is doing (whereas these readable expressions feel much too long)

                            Is

                            >>> path_segment = zero_or_more(none_of('/'))
                            >>> path = zero_or_more(text('/') + path_segment)
                            

                            really more readable than

                            ( / [^/]* )*
                            

                            ?

                            Edit: nevertheless you can create unreadable (and unmaintainable) regexen pretty easily. So I shall add that I would strongly recommend to create “building blocks” of regular regexen, like in this example from stackoverflow

                                    (?<value>           (?&quoted_value) | (?&unquoted_value) )
                            
                                    (?<unquoted_value>  [\w:\-] *                             )
                            
                                    (?<any_quote>  ["']      )
                            
                                    (?<quoted_value>
                                        (?<quote>   (?&any_quote)  )
                                        (?: (?! \k<quote> ) . ) *
                                        \k<quote>
                                    )
                            
                            1. 5

                              I’ll grant you that for short expressions, the standard syntax is generally more readable. That said, your example wouldn’t be equivalent under most implementations (without flags at least, because of the whitespace) and the problem that this library solves is more that of composability rather than readability.

                              I got the idea for the library while reading this tweet: https://twitter.com/joeerl/status/1048483192330706944

                              Here’s how that might look like using cursive_re:

                              regexp = compile(
                                  text("[") + group(zero_or_more(none_of("]"))) + text("]") +
                                  text("(") + group(zero_or_more(none_of(")"))) + text(")") +
                                  group(zero_or_more(anything()))
                              )
                              

                              A lot more verbose, but it’s also a lot more clear what’s going on because the meat of the expression is not obfuscated by escapes.

                              1. 1

                                regexen

                                I beleive it is spelled properly as regexii

                              1. 8

                                I just looked at the docs and I liked a lot of the ideas (the interface (ish) class based views, for instance), but the idea of passing mutable request and response objects and not returning anything really rubes me the wrong way, for some reason I can’t properly explain yet.

                                I also like how it supports async, though, and has a built-in solution to production deploy as well.

                                1. 4

                                  A lot of HTTP framework start with the end-point taking a request as input and return the response. This works beautifully until streaming responses need to be used.

                                  In reality a HTTP request looks more like this (read headers) -> (maybe read request body) -> (send headers) -> (maybe write response body). If the response object is returned it’s hard to control what is going into the body. It could be a long-poll for example. Or streaming from another data source and with a cancel event. There are a lot of scenarios here.

                                  1. 2

                                    I’m generalising, but fewer of your endpoints are going to need streaming so why would you hurt the common case in favour of an uncommon one?

                                    In molten I’ve implemented streaming via generators, which is exactly the opposite approach: all endpoints are functions of a request that return a response and streaming endpoints are just a special case over response types.

                                    1. 1

                                      If it’s a “HTTP Service Framework” I expect it to handle more complicated things than render HTML pages.

                                    2. 1

                                      Hmm, I see. Makes sense. So, probably the request object is not fully loaded, then? As in, the body hasn’t been read, it will be read only on demand?

                                      Anyway, now I’m curious to see how this could play with async/await =)

                                      1. 1

                                        I’m curious too, I am mainly reflecting my experience of using Go

                                      2. 1

                                        This works beautifully until streaming responses need to be used.

                                        It can work well even then. You just need to have a mechanism that allows attaching a generator function as the response body (framework checks response body for callable perhaps?).

                                        psuedocode-ish:

                                        @api.route("/hodor")
                                        def get_hodor(req):
                                            def somefunc():
                                              for x in range(1,100):
                                                yield "hodor"
                                                sleep(1)
                                        
                                            resp = make_response()
                                            resp.body = somefunc 
                                            return resp
                                        
                                          1. 1

                                            Sure it can work but it becomes quickly awkward when mixing multiple streams together in my opinion. Now half of the handler logic is running in another function. I prefer to handle all of the logic in the handler body, this also allows for better destructor (close) handling.

                                            Let’s say I want to implement a Server-Sent Event endpoint. It needs to handle the following cases:

                                            • subscribe to a redis channel, if another request is already using that channel, share the subscription
                                            • if the redis connection gets disconnected, re-connect without losing the client
                                            • unsubscribe if all clients are gone
                                            • send keep-alive if no message has been sent for 60 seconds
                                      1. -6

                                        It’s literally called 0.19. Do you know what a zero at the beginning of a version number means? Christ.

                                        EDIT: more nuanced view below. I stand by what I said here though.

                                        1. 6

                                          The issue is nothing in the way the language is marketed points at this.

                                          You can start at http://elm-lang.org/ and end up installing the language before you even find out what version you’re installing and there’s certainly no mention of how unstable the language is release-to-release, not from the website, nor from its proponents. It’ll also take you a while before you discover its more complex limitations (e.g. the things you used to need modules for) and by that time you may already be heavily invested.

                                          1. 3

                                            That is fairly bad. I’d hesitate before saying that people shouldn’t make pretty websites for things that are still in development, but they should certainly point out that they’re still in development.

                                            Still, it looks like this is less an issue of being unstable and more an issue of people relying on implementation details. Changing implementation details marked as being implementation details you shouldn’t rely on isn’t really that bad. It’s certainly not bad for the web. In the Linux kernel it’d be a sin. I guess a language falls more on the kernel side than the average web library side of being careful about stability.

                                        1. 5

                                          I find it interesting that the author uses AST rewriting to convert |. I’ve seen approaches in the past which just use a filler object (like foo | into | bar, into being the filler) which has a special __or__ method that does the piping.

                                          I personally have experimented with a simple function just called pype() which also does some magic to avoid the need for lambda functions, see here.

                                          EDIT: I did a quick writeup of my solution.

                                          1. 7

                                            I always just reduced the value over the reverse of callables. It made higher order stuff, particularly data transforms, a lot easier to read. The code I used at my last job is open sourced here (the thread() function): https://github.com/curiosity/yunobuiltin/blob/develop/yunobuiltin.py#L444

                                            We used pipeline() more than thread(), which takes only callables (no value) and returns a callables. It is functional composition with the order reversed such that the callables are applied from left to right.

                                            The nice advantage of using the ast is that you can get much closer to Clojure’s -> and friends. By working from a compiler hook (the ast transformer), you wouldn’t need to use partial/rpartial/lambda as much. The downside is that it is quite a bit more complicated than a one liner.

                                            1. 2

                                              That is very nice and clean, though I don’t really like lambdas in Python, they feel super verbose for what they are. I wish there was Haskell-style partial application.

                                            2. 6

                                              I’d considered that approach as well, but I wanted to avoid needing a wrapper for the leftmost expression and I also wanted to avoid needing to partially apply functions.

                                              Take the 3rd example from the post:

                                              Users.find_all() | group_by_category(max=5) | print()
                                              

                                              without transforming the ast that’d have to look something like:

                                              pipe(Users.find_all()) | partial(group_by_category, max=5) | print
                                              
                                            1. 5

                                              $HOME – I released a new version of molten this weekend. The highlight being websocket support; I don’t really need websockets for anything in particular but I realised I’d never read the spec before so my inner yak shaver took over and did what he does best. This week, I’m probably going to give Dramatiq a little love since folks have opened a couple of PRs I haven’t had a chance to look at yet.

                                              $WORK – I’m going to be working on coming up with new ways to make it easier for folks to integrate with our product.

                                              1. 1

                                                Where does molten fit on the flask -> pyramid -> django spectrum of complexity / included batteries?

                                                1. 1

                                                  I would say the core framework is simpler (in the sense of fewer, less coupled components) than flask:

                                                  • API-focused,
                                                  • no external dependencies other than typing-extensions and typing-inspect which are tiny,
                                                  • no global request context,
                                                  • no blueprints,
                                                  • no CLI,
                                                  • no development server,
                                                  • no sessions or templating.

                                                  It does, however, have everything you would need to build and API w/o bringing in other dependencies. The molten.contrib package then lays more functionality (sessions, templates, ORM, etc.) on top of that (and more dependencies!).

                                                  I would probably place it somewhere between flask and pyramid.

                                                  1. 1

                                                    Cool, sounds interesting. I think not having a development server might raise the barrier to entry, though. Every time I’ve started something with flask or pyramid it’s been with the development server. Sometimes that’s even enough for small internal projects or throwaways.

                                                    Blueprints have been one of the most confusing things in flask for me (though it’s been a few years since I tried to use them, maybe they’re better now).

                                              1. 6

                                                I’d use Dramatiq with APScheduler, but I wrote the former so I’m biased!

                                                1. 2

                                                  Was definitely also looking at Dramatiq! Can you explain why one needs both APScheduler and Dramatiq?

                                                  I also nearly missed the Cron-like scheduling in the cookbook when I looked at the Scheduling Messages portion of the guide.

                                                  1. 2

                                                    APScheduler (or something like it) is necessary if you want to execute tasks at regular intervals (i.e. “cron-like scheduling”) since Dramatiq doesn’t have anything built-in to do that. Of course, you can also just use cron instead of APScheduler. :D

                                                1. 9

                                                  Current job (small team):

                                                  • Commit and push changes
                                                  • Concourse CI picks those changes up and
                                                    1. if they’re not tagged, it runs the unit tests and stops
                                                    2. if they’re tagged, it runs the unit tests and continues
                                                  • It builds docker images and pushes them to ECR
                                                  • It deploys those images to ECS in the staging environment
                                                  • We monitor the changes in staging (metrics are scraped by Prometheus w/ graphing from Grafana)
                                                  • If everything looks good in staging we push a button in Concourse to send the images to ECS in the prod environment

                                                  Concourse build pipeline definition, tasks and scripts are defined in the repo and infra is managed with Terraform (which Concourse runs). It took me about three days to set everything up and it has been running smoothly ever since (~6 months).

                                                  To roll back, we just re-run older deployment jobs.

                                                  We don’t have a dedicated QA team. Everyone tests their own changes or asks someone else on the team for help and we have an extensive unit test suite.

                                                  Side projects (just me):

                                                  • ./scripts/deploy runs tests and deploys either to GAE or Heroku and that’s it.
                                                  1. 1

                                                    Same, but Jenkins and Datadog. It just works.

                                                    1. 2

                                                      Oh, using the tags to decide whether to deploy or not is a niiiice idea, I’ll steal it.

                                                  1. 4

                                                    I’m Bogdan and I sometimes write about Python.

                                                    Link: https://defn.io

                                                    1. 5

                                                      Improving the docs and releasing molten 0.2.0. The main focus of this release is going to be Python 3.7 and OpenAPI support (the framework now bundles the Swagger UI and can automatically generate OpenAPI documents).

                                                      1. 3

                                                        Molten looks neat. Especially request validation based on schema classes looks like it would eliminate so much validation code. Nice work!

                                                        1. 2

                                                          Thanks!

                                                      1. 2

                                                        I played around with it and molten is awesome. Still not sure, whether type hints are pythonic or not, but this gave me another tilt towards finding it perfectly suitable, especially for the web dev use case.

                                                        1. 1

                                                          This specific usage of the type annotations is nice in terms of visual clutter, but I think that it will require extra cognitive work to understand and write the code.

                                                          1. 4

                                                            I disagree, type hints are awesome especially for things like this.

                                                            Usually, when your route takes e.g. a user_id you either have to use the right regex (in Django’s urls.py) or remember to validate it yourself in every route. Screw up once -> boom.

                                                            Molten, similar to Rocket for Rust, makes sure that you’re handler is only called when the passed user_id is a valid integer (assuming user_id: int).

                                                            This becomes even more valuable with more complex types.

                                                            1. 1

                                                              I agree that type hints can be useful to make sure that these kinds of bugs are taken care of easily. But the way it is used in the Todo example is that the TodoManager class is passed as an annotation in the methods “list_todos” and “get_todos” . Why not just use a keyword argument for this?

                                                              1. 1

                                                                Why not just use a keyword argument for this?

                                                                That “just” makes the problem seem less complicated than it really is! :D

                                                                Let’s say you’re using a framework like falcon that doesn’t provide DI support out of the box. Assuming you wanted to

                                                                1. be able to swap out the implementation of todo manager easily in your tests and
                                                                2. avoid global state and mocking,

                                                                then your API might look something like this:

                                                                from falcon import API
                                                                
                                                                # Define Database and TodoManager here...
                                                                
                                                                class TodosResource:
                                                                  def __init__(self, todo_manager):
                                                                    self.todo_manager = todo_manager
                                                                
                                                                  def on_get(self, req, resp):
                                                                    resp.media = self.todo_manager.get_all()
                                                                
                                                                
                                                                def setup_app(db_impl=Database, todo_manager_impl=TodoManager):
                                                                    db = db_impl()
                                                                    todo_manager = todo_manager_impl(db)
                                                                
                                                                    app = API()
                                                                    app.add_route("/todos", TodosResource(todo_manager))
                                                                    return app
                                                                
                                                                # In your entrypoint:
                                                                app = setup_app()
                                                                
                                                                # In your tests:
                                                                app = setup_app(db_impl=StubDatabase)
                                                                

                                                                This isn’t bad and it’s explicit, but manually configuring the dependency tree can get quite hairy over time and when a particular component’s dependencies change, rather than only changing that component’s code to depend on different deps, you’ll have to also change all of the places where the dependency tree is constructed (not a huge deal if you have this sort of setup_app function like I’ve got above, but not a particularly fun thing to do either way). Another downside here is that it’s not immediately clear what components the TodosResource.get method relies on from its header.

                                                                If, instead, you use type annotations to do the wiring for you then you get the best of all worlds in exchange for a little bit of magic (that, in my experience, people get used to fairly quickly).

                                                                I hope that makes sense!

                                                              2. 1

                                                                Yeah, I was reminded a lot of Rocket when I was reading the examples. Which is a good thing. There are a lot of things in Rocket I wish I was able to do more transparently in Django.

                                                          1. 3

                                                            This looks interesting, and makes good use of typing. It also looks similar to Apistar, which is another python framework for APIs that also uses typing. Could you please describe the differences between these two? I’ve been using Apistar for a project for a while and it works great so far.

                                                            1. 3

                                                              Thanks! I tried to do that on the motivation page. Basically, I wanted more control over the component lifecycle (molten lets you define which components are cacheable (reused during a single request) and which are singletons), I think apistar’s hooks are super clunky and, as an apistar user myself, this announcement kind of scared me, especially after the huge 0.3 to 0.4 transition.

                                                              1. 3

                                                                I’m sorry, I was so amazed by all the examples on the homepage that I’ve missed the “motivation” link.

                                                                That explains it to me, thank you. I also wasent aware of the change of focus re. Apistar, I’m still running 0.3 but I share your concern.

                                                                I’ll definitively have a thorough look at Molten and will give porting my API over to it a try.

                                                            1. 5

                                                              Coming from the DM, I had a hard time finding the gofundme link. Just in case others are as dense as I am, here it is: https://www.gofundme.com/lobsters-emoji-adoption

                                                              1. 2

                                                                It took me a moment to realize the link is the story link for this story. I’m just not used to seeing the announce tag on a link!

                                                                1. 2

                                                                  Or on a primarily text post!

                                                              1. 5

                                                                @home: Preparing to release version 0.12 of Dramatiq. I want to improve some parts of the docs and some of the supporting material. I may even do a little intro screencast.

                                                                EDIT: I ended up recording that screencast 🎉

                                                                1. 2

                                                                  nice job on the recording, I have tried a few in the past and was never satisfied with the result. What recording software/hardware do you use?

                                                                  1. 3

                                                                    Thanks! I used ScreenFlow to record the video and Moom to size up the windows beforehand to 1280x720pt (2560x1440px) and many, many takes. When exporting the video I chose to do it at 1440p to avoid scaling artefacts. For the audio I used my apple earbuds in a fairly tall, echoey room and it shows :D

                                                                1. 3

                                                                  @home: working on a Django app/integration for Dramatiq. It’ll have a run command and task module auto-discovery built-in, as well as an optional admin interface to manage tasks.

                                                                  @work: pairing w/ some people to improve our local dev. story.

                                                                  1. 1

                                                                    What exactly do you mean by “local dev story”?