“without a ledger.”

That part sounds refreshingly different after all the blockchain startups in that space.

It depends on what you’re trying to deploy and what constraints you have; there isn’t one magic bullet. One pipeline I was especially proud of for a Python app I wrote at Fog Creek worked like this:

1. Create a pristine dump of the target version of the source code. Say the revision is 1a2b3c4d. We used Mercurial, not Git, so the command was hg archive -t tbz2 -R /path/to/bare/repo -r 1a2b3c4d, but you can do the same in Git.
2. Upload this to each server that’ll run the app, into /srv/apps/myapp/1a2b3c4d
3. Based on the SHA1 of requirements.txt, make a new virtualenv if necessary in /srv/virtualenvs/<sha1 of requirements.txt> on each server hosting the app.
4. Copy general configuration into /srv/config/myapp/1a2b3c4d. Configs are generally stored outside the actual app repo for security reasons, even in a company that otherwise uses monolithic repositories, so the SHA here matches the version of the app designed to consume this config info, not the SHA of the config info itself. (Which should also make sense intuitively, since you may need to reconfigure a running app without deploying a new version.)
5. Introduce a new virtual host, 1a2b3c4d.myapp.server.internal, that serves myapp at revision 1a2b3c4d.
6. Run integration tests against this to make sure everything passes.
7. Switch default.myapp.server.internal to point to 1a2b3c4d.myapp.server.internal and rerun tests.
8. If anything goes wrong, just switch symlinks of default.myapp.server.internal back to the old version.

Now, that’s great for an app that’s theoretically aiming for five-nines uptime and full replacability. But the deploy process for my blog is ultimately really just rsync -avz --delete. It really just comes down to what you’re trying to do and what your constraints are.

I doubt you’ll find consistent views, which makes that the opposite of a stupid question.

My ideal deployment pipeline looks something like the following:

• Deployable artifacts are built directly from source control by an automated system (e.g. Jenkins).
• Ideally, some sort of gate is in place to ensure code review has occurred before a deployable artifact is built (e.g. Gerritt, though that project is very dogmatic and while I don’t disagree with it, I don’t strongly stand behind it either).
  • CI builds off unreviewed commits are fine, but I would consider them a developer nicety, not a part of the deployment pipeline.
• Deployable artifacts are stored somewhere. Only the build tool should be able to write to it, but anyone should be able to read from it. (I don’t care what this looks like. Personally, I’d probably just use a file server.)
• Deployment into a staging environment is one-click, or possibly automatic, from the artifact store.
• Deployment into a production environment is one-click from the staging environment. The application must have successfully deployed into staging to be deployed into prod. Ideally, the application must go through some QA in staging to be deployed to prod, but that’s a process concern more than a technical one.
  • Operational personnel need to be able to bypass QA (in fact, bypass almost all of this pipeline) in outage situations.

Note that I’m coming at this from the server-side perspective; “deploy into” means something different for desktop/client software, but I think the overall flow should still work (though I’ve never professionally developed client software, so I don’t know for sure).

We do:

1. Tests are ran, gpg signatures on commits checked, we are ready to deploy!
2. Create a clean export of the revision we are deploying: hg archive -r 1a2b3c4d build/
3. Dump the revision # and other build info into a version file in build/ this is in JSON format as a {}.
4. Shove this into a docker image: docker build -t $(JOB_NAME):$(BUILD_NUMBER) . and push it to internal docker registry.
5. Update nomad(hashicorp product) config file to point to the new $(BUILD_NUMBER) via sed: sed -e "s/@@BUILD_NUMBER@@/$(BUILD_NUMBER)/g" $(JOB_NAME).nomad.sed >$(JOB_NAME).nomad.
6. Do the same as previous step but for the environment we will be running in (dev, test, prod) if required.
7. nomad run $(JOB_NAME).nomad

Nomad will handle dumping vault secrets, config information, etc from the template directive in the config file. So Configuration happens outside of the repo, and lives in Vault and Consul.

You can tell by the env. variables we use Jenkins :) different CI/CD systems will have different variables probably. If unfamiliar with Jenkins, BUILD_NUMBER is just a integer count of how many builds jenkins has done for that job. JOB_NAME is just the name you gave it inside of Jenkins for this job.

What does a great deployment pipeline look like?

I do a “git push” from the development box into a test repo on the server. There, a post-update hook checks out the files and does any other required operation, after which is runs some quick tests. If those tests pass, the hook pushes to the production repo where another post-update hooks does the needful, including a true graceful reload of the application servers.

If those tests fail, I get an email and the buggy code doesn’t get into production. The fact that no other developer can push their code into production while the codebase is buggy is considered a feature.

Since I expect continuous integration to look like my setup, I don’t see the point of out-of-band testing that tells you that the code that reached production a few minutes ago is broken.

The setup we use is not even advanced, but simply resilient against all the annoyances we’ve encountered over time running in production.

I don’t really understand the description underneath “the right pattern” of the article. It seems weird to have a deploy tree you reuse everytime?

Make a clean checkout everytime. You can still use a local git mirror to save on data fetched. Jenkins does this right, as long as you add the cleanup step in checkout behaviour.

From there, build a package, and describe the environment it runs in as best as possible. Or just make fewer assumptions about the environment.

This is where we use a lot of Docker. The learning curve is steep, it’s not always easy, there are trade offs. But it forces you to think about your env, and the versions of your code are already nicely contained in the image.

(Another common path is unpacking in subdirs of a ‘versions’ dir, then having a ‘current’ symlink you can swap. I believe this is what Capistrano does, mentioned in the article. Expect trouble if you’re deploying PHP.)

I’ll also agree with the article that you should be able to identify what you deploy. Stick something produced by git describe in a ‘version’ file at your package root.

Maybe I’m missing a lot here, but I consider it project specific details you just have to wrestle with, in order to find what works. I’ve yet to find a reason to look into more fancy stuff like Kubernetes and whatnot.

I think that the point kaiju’s thread wants to make is that you shouldn’t be deploying from your local machine, since every developers environment will differ slightly and those artifacts might cause a bad build when sent to production. I believe the normal way is to have the shared repo server build/test and deploy on a push hook, so that the environment is the same each time.

This article ( http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html) is quite recent, and refers back to the link here, so I’m guessing this is what prompted this post?

I’d love to hear your conclusions from your climate-change impact investigations whenever you’ve made them :) This is also something I’m interested in.

Ah, another Javascript developer I see!

I’m here to listen if it’ll help.

You mean

# e.g. "col1\tcol2\n    ^ woah" => "Col1 Col2 ^ Woah"

Naming it hurts in this case, because the function does not do what you named it (e.g. in a string of tab-separated values, or a string where multiple spaces are used for formatting). If you had to name it, it would be better named as split_on_whitespace_then_upcase_first_letter_and_join or leave it unnamed and hope that everyone on your team knows that split in Ruby doesn’t work as expected.

The best solution is one that embodies exactly what you intend for it to do, i.e. substitute the first letter of each word with the upper case version of itself. In Ruby, that would be:

sentence.gsub(/(\b.)/) { |x| x.upcase }

Among other benefits, giving it a name means we can explode the code without worrying about a few extra lines in the middle of the caller.

words = sentence.split ' '
words.each { |w| w[0] = w[0].upcase }
sentence = words.join ' '

Introducing a variable called ‘words’ is a solid hint about the unit we’re working with. We may not want to pollute the caller with a new variable, but in a subroutine that’s not a problem.

Naming it does help in this case, but mostly because the reader no longer has to scrutinize over what it’s actually doing. Isn’t this sort of like polishing a turd?

That only masks the issue.

Any maintenance on that line will still have the same problems, whereas refactoring it to split it up into smaller segments AND giving it a name avoids that issue.

A kind of offtopic question based on this comment.

Would I use Coq to prove this function?

Might be worth checking out Gatling for web performance testing. I’ve had good experience with it.

I got a little raise at work last week so lemme share the lobste.rs love. Check your e-mail!

It’s pay what you want for the first set of books, or $15 for all of them. Heck, I feel like I shouldn’t say this, and I don’t know you, but if you really have <$15 and promise to read at least part of each book I’ll buy you the whole bundle if you contact me privately.

Some of the books are released under the Creative Commons and are available online, released for free legally by their authors/publishers. I know Automate the Boring Stuff (https://automatetheboringstuff.com/) and Hacking the Xbox (https://www.nostarch.com/xboxfree) are among those.

That sounds really cool! I’d consider buying a product like that for my workbench at home where I find myself regularly switching between my desktop, laptop, RasPi, and Novena.

I composed an almost identical reply. It’s interesting how apparently this is a rather common story. In my case, this experience, oft repeated during my college career, has given me an almost neurotic relationship to the classroom that haunts me to this day.

Thanks, this looks like exactly the kind of notebook for me.

Yes, it is night and day better in my opinion. LYAH is OK, its not bad by any means but honestly I didn’t learn much from it.

I’d put it in as a nice toe dipping book.

I’m biased, but here’s a post I wrote about learning materials shortly after I started the book: http://bitemyapp.com/posts/2014-12-31-functional-education.html

I’ve given a talk titled “How to learn Haskell in less than five years” as well: https://www.youtube.com/watch?v=Bg9ccYzMbxc