1. 9

In the past I was using drone.io, which didn’t require putting any configuration file in the repository. I simply logged into it, added repository (be it GitHub or BitBucket), configured it (choosing project language and setting triggering options), and finally provided my own build recipe (shell script). It wasn’t the best service out there (had slightly outdated toolchains, etc.), but it was working fine. Wanted to look into it today, but it seems it’s no longer there, or more precisely, something is there, but w/o stuff that was available before, so all links to builds, etc. are no longer valid (404). I hate when services where I have accounts vanish without any prior notice. But ad rem. So…

Are there any continuous integration services (for building and testing software) for open-source projects not requiring configuration file in the root of repository? I’m mostly interested in free ones for public projects.

Ideally I would prefer CI configuration outside of the project’s git repository (let’s call it 1st category), but if it has to be in the repo yet doesn’t have to placed in the root, then it’s also an improvement (2nd category).

Sadly, popular CI solutions (like Travis or Shippable) that support open-source projects (i.e. providing their service to them for free), require some .yml file in the root of repository that you want to run builds and tests on. Most people are apparently indifferent about it, and I may be in very small minority with my view on it, but I simply don’t like polluting repository, especially the root of it, with stuff that could be avoided.

Sometimes some services are not yet well-known, and sometimes some knobs are well-hidden, that’s why I am reaching for collective Lobsters knowledge, hoping to be pointed to some solutions I possibly overlooked or not googled enough.

  1.  

  2. 4

    Out of curiosity, why? I’ve always considered it to be superior to have build configuration versioned directly with code. You can guarantee lockstep changes with build and code and you only need a single to to understand code and how it’s built.

    We use Jenkins with configuration solely in Jenkins at the moment and it’s inferior to if we used Jenkins files in our repos.

    1. 4

      I hope I answered in my other comment, at least to some extent.

      Building (and testing) recipe shouldn’t be part of CI configuration file, it should be part of Makefile or whatever is used for your language toolchain/building framework. You should be able to build application w/o having your own CI instance. Sure, Makefile won’t install packages needed to build software (dependencies should be mentioned in README file), as it’s a distro-specific thing, so that’s why CI environments need preparation step, and it needs to be stored somewhere. I just don’t like storing such things in project’s repository, as it’s strictly related only to tooling wrapping the project, and this tooling may change at some point, it’s CI-specific (and therefore often distro-specific) thing, so it’s meta project workflow thing. If you want it versioned, I think it should be versioned separately.

      If you care about compatibility, you should be able to avoid lockstep changes (your older source code should build fine with newer build preparation configuration or your newer source code should build fine with older build preparation configuration). And if it’s not possible, then it’s good thing to have clear indicator as build failure, because it will most likely hit users too.

      README (or INSTALL) file should be enough to let user/developer know how to prepare building environment in their own distro (which may not necessarily be your distro or distro used by CI).

      tl;dr (simplistic): Today you may be using one CI, but it may be other thing in future. It’s not crucial part of the project (even if very useful), thus it shouldn’t pollute its development repository. That’s my view.

      1. 1

        This is my pet hate with Jenkins, every time we open a new longer-lived branch that deserves a CI build, it’s an exercise in copying config from one browser tab to another. I’d love to just be able to modify a .jenkins.toml file.

        1. 3

          If you use a newer Jenkins with Pipelines, you can use a Jenkinsfile.

      2. 4

        AWS CodeBuild allows you to specify your buildspec file either within the console or using CloudFormation so you can store all of that configuration completely outside of the repository. It’s not technically a free product but if you use 100 build minutes or less a month you’ll fall into the free tier, past that it’s still pretty cheap at $0.30 per build-hour. One other nice feature is that, while they provide build images, you can bring your own if you have very specific toolchain or version needs.

        1. 2

          It requires to connect to SCM on following terms:


          GitHub

          Repositories: Public and private

          This application will be able to read and write all public and private repository data.

          Organizations and teams: Read-only access

          This application will be able to read your organization and team membership.


          BitBucket

          AWS CodeBuild … is requesting access to the following:

          • Read your account information
          • Read your repositories

          so it’s much saner here.

          But there is also a warning:

          AWS CodePipeline does not support Bitbucket.

          AWS CodePipeline cannot build source code stored in Bitbucket. If you want AWS CodePipeline to use this build project, choose a different source provider.

          Not sure yet what it exactly means, i.e. how CodeBuild and CodePipeline are related.

          1. 2

            By default only Ubuntu 14.04 is available:

            [Container] 2017/12/10 00:53:59 Running command uname -a
            Linux 9d4937a7a169 4.9.58-18.55.amzn1.x86_64 #1 SMP Thu Nov 2 04:38:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            
            [Container] 2017/12/10 00:53:59 Running command lsb_release -drc
            Description:	Ubuntu 14.04.5 LTS
            Release:	14.04
            Codename:	trusty
            
            [Container] 2017/12/10 00:54:00 Running command gcc -v
            Using built-in specs.
            COLLECT_GCC=gcc
            COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
            Target: x86_64-linux-gnu
            Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.8.4-2ubuntu1~14.04.3' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
            Thread model: posix
            gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
            

            Whole “building” (running these commands) took around ~7 seconds, but most of time was spent before calling these commands:

            [Container] 2017/12/10 00:53:53 Waiting for agent ping
            [Container] 2017/12/10 00:53:54 Waiting for DOWNLOAD_SOURCE
            [Container] 2017/12/10 00:53:58 Phase is DOWNLOAD_SOURCE
            [Container] 2017/12/10 00:53:59 CODEBUILD_SRC_DIR=/codebuild/output/src906734987/src/...
            [Container] 2017/12/10 00:53:59 YAML location is /codebuild/readonly/buildspec.yml
            [Container] 2017/12/10 00:53:59 Processing environment variables
            [Container] 2017/12/10 00:53:59 Moving to directory /codebuild/output/src906734987/src/...
            [Container] 2017/12/10 00:53:59 Registering with agent
            [Container] 2017/12/10 00:53:59 Phases found in YAML: 1
            

            Phase details:

            • SUBMITTED Succeeded
            • PROVISIONING Succeeded 24 secs
            • DOWNLOAD_SOURCE Succeeded 4 secs
            • INSTALL Succeeded
            • PRE_BUILD Succeeded
            • BUILD Succeeded 1 sec
            • POST_BUILD Succeeded
            • UPLOAD_ARTIFACTS Succeeded
            • FINALIZING Succeeded 3 secs

            But other docker images can be indeed provided too. Will check it later.

            1. 2

              All of the Dockerfiles for their provided containers are open-sourced on GitHub. I’ve used them as a jumping-off point for building custom images with good luck in the past.

              They’ve also announced Windows support is forthcoming and you can sign-up for early access. I haven’t played with it though since I don’t have any projects that require Windows.

              1. 1

                Thanks for the links. It’s good to know how exactly their docker images are created. Windows support is also useful feature for cross-platform software.

            2. 2

              They also support zip files in S3, so if you had a use-case that didn’t work with either GitHub or BitBucket you could just zip up your source and build it like that. The bit about CodePipeline only applies if you’re using CodePipeline to orchestrate a build and deployment workflow, you can also just use CodeBuild standalone which removes those limitations.

              1. 1

                They do support S3, but it’s amazon’s service, so I couldn’t expect less from them. Jokes aside, I’m not S3 user, but it may be a useful feature for some workflows out there in the wild. CodePipeline has only 1 active pipeline per month in free tier, which I presume could be a problem in case of wanting to use it in more than one project.

                1. 2

                  I’ve only used the S3 support a few times but it comes in handy for any really odd build workflows that are tough to model in a repository alone or require some form of external input, such as updating docker images based on changes to the dependent software within that container (following Linux package updates, etc…) but I consider those flows to be a bit of a hack.

                  Where I’ve found that CodePipeline really shines is when you need to deal with builds and complex promotions to testing environments then eventually to production. That being said I really use it for a few of my side projects. For the basic build, test, publish flow it’s easy enough to model in CodeBuild alone.

            3. 1

              Interesting! Didn’t know that AWS provides something like this. Cannot comment on CodeBuild pricing, as I have never compared prices for build services before. But 100 minutes should be enough for smaller projects without thousands of commits per month.

            4. 3

              IIRC codeship doesn’t store the build configuration in the repo, it stores the build config in your project on their site. It was kinda nice that I didn’t have to push commits to iterate over the various quirks in the build environment.

              I’m pretty sure codeship is not only free for open source projects but even a limited number of private ones too?

              1. 1

                Thanks for mentioning Codeship!

                It seems that in their Basic version they don’t require any configuration file. You configure your commands via web UI. Not sure about private ones yet, as their pricing page doesn’t mention anything about it.

                1. 2

                  Yes, and they allow you to use ssh to debug failures or investigate/prototype.

                  I’m almost certain that I have used them in the past with a private bitbucket repo.

                  1. 3

                    I see one problem with Codeship for starters, though. They don’t allow you to add a project without connecting your SCM first, which means granting a lot of permissions to codeship, i.e. seemingly more than truly necessary.


                    GitHub

                    Personal user data:

                    • Email addresses (read-only)

                    Repositories: Public and private

                    • Code
                    • Issues
                    • Pull requests
                    • Wikis
                    • Settings
                    • Webhooks and services
                    • Deploy keys
                    • Collaboration invites

                    BitBucket

                    • Read and modify your account information
                    • Read and modify your repositories’ issues
                    • Access your repositories’ build pipelines and configure their variables
                    • Read and modify your team’s project settings, and read and transfer repositories within your team’s projects
                    • Read and modify your repositories and their pull requests
                    • Administer your repositories
                    • Delete your repositories
                    • Read and modify your snippets
                    • Read and modify your team membership information
                    • Read and modify your repositories’ webhooks
                    • Read and modify your repositories’ wikis

                    GitLab

                    • Access the authenticated user’s API:
                      Full access to GitLab as the user, including read/write on all their groups and projects

                    Second problem is similar to what was I had with old drone.io: old preinstalled packages. I’m mostly referring to very old GCC 4.8 here.

                    And I’m still not even sure that building C/C++ projects is actually really supported here, as it’s not mentioned explicitly anywhere.

                    1. 1

                      Great points, I used it for Python w/o any native/extension code. It was fine for that but might not be suited for C/C++.

              2. 3

                For travis, there is the way of having a versioned repo with a travis.yml, which pulls in your other repo (in whatever fashion you prefer) and builds it.

                1. 1

                  Right, that’s also a possibility. It may not work in the case of CI downloading repo and disallowing any external network access. (Theoretically git submodule feature could help here, but it ties you to given commit or branch, which makes it somewhat less flexible.)

                  The upside is that build/test CI configuration for project is versioned separately from the project itself.

                  The downside is that you lose features related to tracking many branches etc., because you connected project-meta-travis repo to Travis CI and not the actual project’s repo, where development happens.

                  Or maybe you meant some other solution? Regardless, feel free to provide examples.

                2. 3

                  You can try SemaphoreCI – they have a free plan and don’t require a configuration file.

                  1. 1

                    Indeed, SemaphoreCI seems to support customizing build commands w/o putting any stuff in the repo, they even recognize possible need to add custom configuration files that shouldn’t be part of the repo.

                    Their C/C++ toolchain is dated, though. GCC 4.9 is the latest one there.

                    Thanks!

                    1. 1

                      It is also the most beautifully designed from what I’ve seen <3

                      1. 1

                        SemaphoreCI is the first service, where you can choose how you to connect to GitHub:

                        • Only Public repos,
                        • All repos, Public & Private.

                        I don’t have private repos on GitHub, but I value greatly that kind of choice. I’m assuming the first option below.


                        GitHub

                        Semaphore by renderedtext wants to access your account

                        Repositories
                        Public only

                        This application will be able to read and write all public repository data. This includes the following:

                        • Code
                        • Issues
                        • Pull requests
                        • Wikis
                        • Settings
                        • Webhooks and services
                        • Deploy keys

                        BitBucket

                        Semaphore is requesting access to the following:

                        • Read your account information
                        • Read and modify your repositories and their pull requests
                        • Administer your repositories
                        • Read and modify your repositories’ webhooks

                        BB permissions aren’t as bloated as in case of Codeship, and are generally more slim than those in CircleCI, but there is one important exception, SemaphoreCI wants to be able to modify pull requests, while CircleCI is fine with only reading them.

                      2. 2

                        Gitlab doesn’t require its .gitlab-ci.yml to be in the root directory. You can change where the CI running looks for .gitlab-ci.yml in the settings for the repo or even change its name entirely.

                        1. 1

                          GitLab CI requires you to host code in GitLab repo, if I’m not mistaken, but it’s still nice that you can use custom CI config path there. (If they already store somewhere meta-config for config location, then it shouldn’t be too hard to store whole config outside of the repository, so maybe it will be added in future.)

                        2. 2

                          I run a Buildbot cluster for an open source project I host for pretty much this reason, and needing to build on multiple platforms which most of the hosted solutions don’t do.

                          I’d be really interested in hosted solutions which can do multiple platforms.

                          1. 2

                            You can configure CircleCI v1 through the web interface, and CircleCI v2 is configured via .circleci/config.yml.

                            1. 1

                              Thanks!

                              It seems a bit more complicated, though. CircleCI 1.0 configuration docs state at the very beginning that:

                              CircleCI automatically infers settings from your code, so it’s possible you won’t need to add any custom configuration. If you do need to tweak settings, you can create a circle.yml in your project’s root directory and CircleCI will read it each time it runs a build.

                              It doesn’t state anything about being able to provide such configuration w/o putting it in the repo. And there aren’t many details about this inference either.

                              1. 1

                                I don’t like that you cannot create standalone account in CircleCI. You have to sign-up via GitHub, BitBucket or Google. Even if you sign-up via Google, you have to connect to GitHub or BitBucket if you want to start building anything.


                                GitHub

                                CircleCI by circleci wants to access your account

                                Personal user data
                                Email addresses (read-only)

                                This application will be able to read your private email addresses.

                                Repositories
                                Public and private

                                This application will be able to read and write all public and private repository data. This includes the following:

                                • Code
                                • Issues
                                • Pull requests
                                • Wikis
                                • Settings
                                • Webhooks and services
                                • Deploy keys
                                • Collaboration invites

                                BitBucket

                                CircleCI is requesting access to the following:

                                • Read your account information
                                • Read your team’s project settings and read repositories contained within your team’s projects
                                • Read your repositories and their pull requests
                                • Administer your repositories
                                • Read and modify your repositories
                                • Read your team membership information
                                • Read and modify your repositories’ webhooks
                              2. 2

                                Most people are apparently indifferent about it, and I may be in very small minority with my view on it, but I simply don’t like polluting repository, especially the root of it, with stuff that could be avoided.

                                How do you feel about a Makefile in the root?

                                1. 1

                                  For software projects Makefile (or any other build recipe used in given language/framework) is crucial part of it, because source code that cannot be build (ideally in one command invocation), is not that useful, to say the least. That’s why I believe its place is exactly in the root of repository. Such Makefile usually provides also a way to start tests or other useful actions. I’ll only add that I find putting Makefile in src/ only (if source files have their own directory) as a bad practice.

                                  I do not find putting CI configuration in the root of repository as necessarily a bad practice, but I don’t like untidiness it brings because of mixing project’s files with 3rd party services’ files. 3rd party services’ stuff is not strictly a part of the project, it’s part of meta project workflow - it can aid project development in general, but is useless if you don’t have guaranteed access to these services (because you’re offline right now, or service stops being available like old drone.io) - that’s why it shouldn’t be in the repo. I find it a bit more tolerable for corporate environment in its internal projects repositories, but only if the company itself hosts and maintains CI tools.

                                  You can sensibly develop, build and test project w/o your-ci.yml file yourself (assuming sensible Makefile or whatever is used by it), yet you cannot sensibly develop, build and test project having your-ci.yml file w/o Makefile/Rakefile/whatever (well, you actually can, but you’ll end up recreating build and test recipe within this CI configuration file only, which is stupid, and you’ll require devs to have their own CI instances, as development per se shouldn’t require on-line access).

                                  Not sure if I’m able to convince anyone, in the end it’s just a matter of one’s view.

                                  1. 2

                                    I agree with you on part of where CI configuration belongs. Making it part of the core repository leads to problems, especially because for some CI tools the only debugging mechanism is “change CI config, commit and push, then watch the job”. And I can’t imagine why anyone would do that…

                                    1. 2

                                      For software projects Makefile (or any other build recipe used in given language/framework) is crucial part of it, because source code that cannot be build (ideally in one command invocation), is not that useful, to say the least.

                                      I could not agree more with this statement.

                                      Reading the rest of your position, I believe my position toward checking in CI configuration is rooted in my use of CI as the base of implementing CD. I see the entire pipeline to production as a single system, and configuration / code need to be versioned as a single unit. The configuration then ends up less what’s needed to build the code— that’s just one part— but also deploy / test / monitor / etc.

                                      A lot of organisations don’t host anything they don’t need to host. Some codebases can’t even be developed without an Internet connection. My Grandmother told me that if I can’t say anything nice, say nothing at all. So I’ll leave it at that.

                                      Thank you for writing up your thoughts so clearly and concisely across this thread! It’s given me much to chew on.

                                  2. 1

                                    Have you considered creating a second repository that uses the first as a submodule?

                                    Your config can live in that repository – it can have a very linearised development, and a post-commit hook in the first repository can update this one.

                                    1. 1

                                      It was even discussed already shortly - see my comment. submodule would make it depend on particular commit or branch, so it would be rather rigid workaround. It’s good to be able to have building & testing available for all branches with one config.

                                      My main concern with your idea is that the first repo (main repo) shouldn’t interact that way with the second one (meta project workflow-related repo). CI should be mostly invisible to the project. Post-commit hook in first repo is acceptable for triggering the build via some API, but using post-commit hook for committing change to other repo, because that would trigger CI already connected to it is… too hacky and unacceptable to my taste.

                                      The only good idea for second repository is just to have versioned changes of build/test preparation configuration, and actual builder should always use latest commit (from master, or any other branch/tag, if that was configurable).

                                    2. 1

                                      Jenkins will do it, but I’m more interested in why you’d want to do this? IMO the ability to co-locate (and version) build instructions with your source code is great boon.

                                      1. 1

                                        I explained it already. Please read this and that.

                                        Is there some free for open-source Jenkins instance out there?

                                        1. 1

                                          Unfortunately, not that I know of, since CloudBees stopped accepting new signups to dev@cloud.

                                      2. 1

                                        @przemoc Hi, I found this thread since you mentioned Shippable… I’m a co-founder :)

                                        You can configure builds on Shippable without putting a shippable.yml at the root of the repository you want to build. While the config is still yml-based, it can live in a separate repository pretty easily. Please reach out to me at manisha@shippable.com and I can point you to a couple of examples of how to do this.

                                        We’re also working on a UI based config which will be launched in a few weeks.

                                        1. 1

                                          I already left similar question to the one I asked here on your page via Conversations box. It was a few days ago and apparently no one was around, so I was asked to provide my e-mail to get an answer later, but I haven’t got any yet. I suggest removing that kind of chat box if you don’t intend to support it.

                                          Good to hear that there is some flexibility in Shippable, because I couldn’t find such information in your docs. It would be great if you could share details here. Or is there any specific reason you cannot write about it publicly?

                                          1. 1

                                            Sorry for the late follow-up from our side to your question on chat. I believe you’re now in touch with our customer success team.

                                            To clarify:

                                            1. 1

                                              No problem, @manishas!

                                              Indeed, I am in touch with one of your colleagues. And so far I am a bit disappointed.

                                              Let’s visit:

                                              There are lacking statuses, even in the latest one commit! And links from those available are wrong, as I get 404!

                                              I archived it just in case:

                                              For public repos such build job logs, that statuses link to, must be always accessible. They are useless otherwise. (For private repos they have to be accessible too, but not necessarily publicly to anyone for obvious reasons.)

                                              1. 1

                                                The issue with broken links to build logs was apparently addressed by Shippable.

                                                So I archived what we have right now in the sample project:

                                                Links are different now, but as older ones gave me 404, new one gives me 404 too.

                                                Previously they looked like this:

                                                so pretty clear.

                                                Now they look like this:

                                                https://app.shippable.com/subscriptions/58b5dd45ddd8e8070045dab1/pipelines/builds/5a30c1fccf141c0700bf6cbe/consoles

                                                which is HUGE UX regression in naming department.

                                                Why so cryptic links for public repos? They should be human-readable.

                                                But most importantly they should be simply working, and be accessible for anyone if it’s public repository.

                                                I’m honestly amazed that service’s basic features seemingly aren’t working properly.

                                                1. 1

                                                  As I said before, this is not the forum to discuss every single email we are exchanging offline.

                                                  If you try our mainline CI workflow, the links are human readable and publicly accessible. For example, here is a link to one of my projects : https://app.shippable.com/github/manishas/basic-node/runs/16/1/console

                                                  I want to again stress that your requirement isn’t a mainstream requirement. UI based config is clunky, un-versioned, and usually considered inferior. YAML based config is becoming the defacto standard, and even traditional tools like Jenkins are moving towards ci config as code.

                                                  I was trying to find a creative way make our platform work for your scenario, but maybe your needs are too specific. Good luck with your search!

                                                2. 1

                                                  @przemoc you’re right that build results for public projects should be public. That is EXACTLY the behavior when you put the shippable.yml at the root of your repo. Most customers want the config yml in the same repository since that keeps source code and build pipeline in the same place.

                                                  The workflow our customer success team suggested is an alternative workflow since you had different requirements and wanted to separate config from source code.

                                                  Anyway, since our customer success team is helping you over email, let’s take it there.

                                                  Overall, I am very disappointed in your communication style. Our customer success lead Ambarish specifically put together a sample for you and has been very responsive in clarifying all questions and issues, including the ones you raise above. I am not sure you have even forked the sample and tried it one time to see if it meets your needs. You should obviously find the best alternative that works for your scenario and needs, but some politeness in the process is always well appreciated.

                                                  1. 1

                                                    Let me respond here to both of your comments in one.


                                                    Reply to 1st comment

                                                    you’re right that build results for public projects should be public. That is EXACTLY the behavior when you put the shippable.yml at the root of your repo.

                                                    So why the behavior isn’t the same when I put the file in other repo? In your first comment on my thread here you wrote:

                                                    You can configure builds on Shippable without putting a shippable.yml at the root of the repository you want to build. While the config is still yml-based, it can live in a separate repository pretty easily.

                                                    So can it live in a separate repository pretty easily or not? Apparently not, because it’s no longer functioning that well. Now I read that:

                                                    The workflow our customer success team suggested is an alternative workflow since you had different requirements and wanted to separate config from source code.

                                                    So why there was no information that build logs won’t be publicly visible from the very beginning?

                                                    Anyway, since our customer success team is helping you over email, let’s take it there.

                                                    I hoped so, but after reading your next paragraph, I couldn’t remain silent.

                                                    Overall, I am very disappointed in your communication style. Our customer success lead Ambarish specifically put together a sample for you and has been very responsive in clarifying all questions and issues, including the ones you raise above.

                                                    I appreciate that some sample was prepared specifically for me to show that what I wanted is possible with Shippable, but in fact it was not working, not even close. When you give someone sample you make sure it works. That’s the whole point of the sample. But let’s look back and see if what you wrote really holds value.

                                                    2 days ago, ~1.5h after getting first mail with the sample from customer success team (which was written to me in response to my question asked via conversation box few days before that), I replied that build links return 404.

                                                    The issue was completely skipped in response from Ambarish (which he sent 3 hours later).

                                                    Next day I replied mentioning another issue (which I didn’t catch before, because I was using smartphone when I was answering the first time, and mobile view on GitHub is limited): there are no commit statuses next to recent commits. And I repeated that I’m getting 404 errors when I try to see build logs from the commit statuses that are present.

                                                    First paragraph of response from Ambarish gave me some hope: “The links to the statuses are broken indeed! We are fixing this bug ASAP. “ 3.5h later there was a follow up message: “We have addressed the issue with the broken link. I had missed the showBuildStatus: true attribute on the gitRepo resource. The commit status links are correct now. You can fork the repo and try it out.”

                                                    Now I thought that maybe my first impression with the sample was wrong and Shippable actually seem to care after all. That’s nice. So I rechecked the repo. Finally there was a commit status next to the latest commit. I hovered over the tick icon and first thing I noticed was horrible and cryptic URL that I mentioned in my other comment already. So I clicked it. And guess what? 404.

                                                    So I replied mentioning that I am still getting 404 error when trying to visit build log. I also complained about needlessly cryptic URL for public repo, as they should be human-readable, and I reiterated: “But most importantly they should be simply working, and be accessible for anyone if it’s public repository.” Lastly, addressing forking invitation I simply stated: “There is no point in me forking the repo, if you cannot show that Shippable works even for some simple sample project.” That may sounded harsh indeed, but how can you react when someone tries to convince you that something works while it doesn’t really. Having build logs and looking into them are basic features in that kind of service for public repositories. At that time I was honestly amazed that they seemingly aren’t working properly.

                                                    In response Ambarish sent me “let me explain the whole story with the timeline”-mail. It sort of explained why commit statuses for older commits had wrong URLs. He ended mail saying that they will fix how URLs to build logs look like and make them more concise and gave me example: https://app.shippable.com/github/ambarish2012/jobs/basicnode_ci/builds/5a30c1fccf141c0700bf6cbe/console

                                                    There was one thing that was sorely lacking in his mail. The matter of 404 error I was still getting for build log for latest commit (e4fa8ae) was not even mentioned!

                                                    I got really irritated (who wouldn’t be at that point?). In the reply I wrote some of my paragraphs looked like this:


                                                    > 3. I then specified  showBuildStatus and the status links started getting
                                                    > posted correctly.
                                                    
                                                    If they're posted correctly now, why they're not working?
                                                    
                                                    404 means PAGE NOT FOUND (I hope you know that common knowledge).
                                                    And I get 404 for clicking the status icon next to your latest commit (e4fa8ae).
                                                    I even provided page archives so it would be clear.
                                                    
                                                        http://archive.is/RXXgC - commits page
                                                        http://archive.is/JjV6I - "build log" that redirects to 404
                                                    
                                                    Have you checked it?
                                                    
                                                    If the link to build log doesn't work, then it's useless as I wrote earlier.
                                                    

                                                    > https://app.shippable.com/github/ambarish2012/jobs/basicnode_ci/builds/5a30c1fccf141c0700bf6cbe/console.
                                                    
                                                    That will be an improvement, definitely. But why basicnode_ci?
                                                    It's true that's where config is placed, but you're building
                                                    basicnode, so it should be reflected in the URL.
                                                    

                                                    At the end I'll repeat what I wrote earlier.
                                                    
                                                    Please fix the links to build logs.
                                                    
                                                    Even bad looking link is better than good looking, which returns 404.
                                                    So far all links are bad.
                                                    

                                                    I'm on the verge of becoming indifferent at this point, because I'm
                                                    mentioning about broken links to build logs from a few mails back and
                                                    they are all still broken, including the latest ones supposedly
                                                    generated after fixing configuration. I don't know why you are unable
                                                    to fix it for that long time. It's core feature, isn't it?
                                                    

                                                    Surely that wasn’t an exemplary mail of politeness, but I was barely keeping my composure in check considering how communication looked so far.

                                                    Today I finally got response:

                                                    I perfectly understand what a 404 is :). The reason the link is a 404
                                                    for you is because you are not a member of my organization in GitHub.
                                                    Basically, I have created a simple pipeline to build my repository and
                                                    to see my pipeline, you need to be a member of my organization.
                                                    
                                                    We can get into the details why you need to be a member of my
                                                    organization to view my pipelines as well if you like.
                                                    

                                                    I’m baffled. Why we’re talking about some organizations all of a sudden? You were building public repo, right?

                                                    So things are apparently much more complex in Shippable than supposed to be and that were presented at the beginning. Well, it could be fine. But!

                                                    • Why not being straight from the beginning?
                                                    • Why no mention that I won’t be able to see build logs from sample application from the very start?
                                                    • And why no explanation what is the reason that you can view build log of public repo if its config is in the repo, but not if the config is in separate repo?

                                                    And regarding this very particular mail strictly, there are questions that immediately pop out after reading it:

                                                    • Why proper answer to my 404 error reports was avoided for so long?
                                                    • Why details of why you need to be a member of organization to view pipeline weren’t shared within this mail?

                                                    Does anyone reading my comment still believes that customer success team was “very responsive in clarifying all questions and issues, including the ones you raise above”?

                                                    I simply cannot agree with it. At the beginning I thought there is a genuine will to help me via showing how Shippable is capable of doing what I want. But with each further mail it seemed like it’s getting dragged for some reason (not in a strictly time-manner, but rather in a way how crucial information was (not) shared and how some of my issues were constantly skipped) and my main problem wasn’t addressed until the very last mail, and in fact only partially, as I am apparently supposed to ask one more time to be explained why it is like it is… And I mentioned so many times in my mails that build logs for public project must be available to anyone that it’s more than obvious that I would like to finally know these damn details, i.e. how is this sample application so different from “Most customers want the config yml in the same repository” case, that it cannot provide public build logs?

                                                    I started providing reports about Shippable in my thread here, because I hoped it would end up with success story - look, they reached me out and showed it can be done with Shippable. So everyone reading the thread in future would know that Shippable cares about their customers and delivers, even to non-paying ones wanting to build open-source projects there. But my hope was premature.

                                                    You say you are very disappointed in my communication style? I’m very disappointed that you sugar-coat how great communication looked from Shippable side.

                                                    I am not sure you have even forked the sample and tried it one time to see if it meets your needs.

                                                    I clearly stated that I didn’t fork it and I also explained why.

                                                    You should obviously find the best alternative that works for your scenario and needs, but some politeness in the process is always well appreciated.

                                                    I was polite, but I don’t deny I got irritated at one point.

                                                    Reply to 2nd comment:

                                                    As I said before, this is not the forum to discuss every single email we are exchanging offline.

                                                    That wasn’t my intention. I do believe, though, that open-source software and services for OSS need to be transparent and open, that’s what makes them best. You showed up in my thread stating that Shippable is up to my needs, so it was only natural that I’ll report back if it really is. I was only reporting here the issues I was facing so far.

                                                    But after seeing your 2 comments today, I decided I’ll share more details, to make the story more complete.

                                                    If you try our mainline CI workflow, the links are human readable and publicly accessible. For example, here is a link to one of my projects : https://app.shippable.com/github/manishas/basic-node/runs/16/1/console

                                                    But I was trying what customer success team prepared for me and that was supposed to match my needs. Talking about your mainline CI workflow here is a diversion.

                                                    I want to again stress that your requirement isn’t a mainstream requirement. UI based config is clunky, un-versioned, and usually considered inferior. YAML based config is becoming the defacto standard, and even traditional tools like Jenkins are moving towards ci config as code.

                                                    I understand that wanting to have configuration outside of main repo that is meant to be built is not a mainstream requirement. Otherwise I wouldn’t post this very question on lobste.rs regarding services that are able to do that.

                                                    And I have no clue why are you now talking about UI-based config. It’s another diversion from you. I prefer text-based configuration hands down, and YAML is fine for that. Have I criticized YAML configuration here or in any mail with Ambarish? I did not.

                                                    I just want the build preparation config for CI service to live outside of main repo, and if that may require using web UI (which actually doesn’t exclude use of YAML there and possibility of being versioned, right?), then I can still consider it despite my fully-text-based config preference.

                                                    I was trying to find a creative way make our platform work for your scenario, but maybe your needs are too specific. Good luck with your search!

                                                    Maybe my needs aren’t as specific as you paint and your platform would work for my scenario, but maybe you failed to communicate it properly, simply sell me on Shippable.


                                                    Maybe I should have accepted your proposal to mail you directly back then, because you seem to know the platform better, but back then I already left a question a few days earlier via your official site conversation box, so I believed I should go the standard way and wait for the reply, to have experience closer to standard prospective customer (who doesn’t know co-founder’s e-mail). I think it’s more fair that way. You were CCed by customer success team from the first mail anyway, so I guess that my experience was supposedly already meant to be better than most others can expect.

                                                    Not even once until today was stated, that supporting config out of the main repo is such a completely different workflow for Shippable, which has its own quirks like lack of publicly visible build logs, even for public repositories. I still don’t know why it is like that. Is it by design? What’s the rationale behind it? Maybe it could be fixed?

                                                    The whole discussion would go completely different if in the first mail from Shippable had proper sample, i.e. all needed information were included:

                                                    • link to the sample main repo with correct commit statuses,
                                                    • link to sample ci repo with correct config from the beginning,
                                                    • extremely important paragraph(s) explaining that to achieve such setup very different workflow (than in case of typical config-in-main-repo) has to be used, because typical run jobs work like this… [explanation], but we need here more advanced pipeline jobs (if I inferred correctly from what I read today from you and Ambarish), which work like this… [explanation], which leads us to the problem, that logs for them, that are linked from commit statuses, aren’t publicly visible, because… [explanation].

                                                    Then my reply would be simply:

                                                    • Can you make logs from pipeline jobs for public repositories publicly visible?
                                                    • Can you make URLs to logs from pipeline jobs less cryptic and human-readable?

                                                    Unfortunately it didn’t go that well.

                                                    I still don’t know answers to many of my questions. My curiosity slightly wants me to continue the mail conversation with Ambarish in hope that eventually they would be answered. My sanity, based on past experience and today @manishas’s comments, objects, though.

                                                    1. 2

                                                      Can you make logs from pipeline jobs for public repositories publicly visible?

                                                      • We’re looking into this issue and will have more information soon. After talking to the team, we need to rationalize how the runSh job type should behave in this situation. As mentioned earlier, the CI job already does this but that doesn’t address your needs.

                                                      Can you make URLs to logs from pipeline jobs less cryptic and human-readable?

                                                      • They are already human readable. The reason the link in Ambarish’s example said ‘basicnode_ci’ and not ‘basicnode’ is because that’s how he named the job that contained the CI commands. If he had named it basicnode, it would have been showed up that way in the URL.

                                                      The runSh job is an Assembly Line job which is not automatically associated with a source code repository, which is why you also have to specify a gitRepo resource as an Input to the job. This is the main reason why I call it ‘not mainstream ci’, since the runSh job was designed to separate repositories from configured jobs.

                                                      As an example, you could potentially configure builds for 10 different projects in a single shippable.yml, along with provisioning jobs, deployment jobs, etc and configure a complete end to end workflow for your application.

                                                      I hope that clarifies why the build link cannot be tied to a separate project and is simply the job name.

                                                      1. 1

                                                        Thank you for your answers, it clarifies some stuff. I really like the flexibility that you provide. I may still consider Shippable for use in the future. Hopefully the matter of publicly visible links for runSh job matters will be fixed by then.

                                                      2. 1

                                                        @przemoc The only reason I reached out is because you mentioned us in the comment.

                                                        Since you’ve taken the trouble of explaining your stance, let me explain mine:

                                                        • I saw that we could satisfy your scenario, and still believe we can. I was proposing a workflow that I wanted you to look at to see if it had what you needed. It’s perfectly ok if you see something that doesn’t work and choose to move on, but as long as we’re communicating, I expect it to be polite.
                                                        • Every time you found something that you either didn’t agree with or in the case of 404 was a bug, your responses were not very polite…. for example, instead of saying: I want builds for public projects to be visible to everyone, or, Why am I getting a 404?, your response seemed to be similar to this : I get a 404!! Even basic services do this and without it it’s all useless, etc etc.
                                                        • These type of statements do not give me confidence that you’re trying to understand why something works a certain way or to point out a bug… it sounds pretty hostile and condescending when in all actuality, we have these features working perfectly for the mainline scenario and were happy to investigate and try to address anything you needed even in the proposed workflow.
                                                        • Your message above is a great example of why I am frustrated. Now you’re accusing me of hiding things or not being upfront. Our customer success process is through a github repository or over email. I cannot come here and keep responding to every comment you post here when the exact same conversation is also happening over email. If your goal was to keep folks updated, you could have completed our conversation over email and posted a recap here at the end of it. Do you really want me to conduct customer success activities here? That’s not a reasonable request and I’ll challenge you to find any SaaS companies that even went to the extent we did.
                                                        • Another great example of your responses when Ambarish forgot to copy me on one email. You could have just copied me again and moved on. Instead your first instinct is to assume he either doesn’t understand or did it on purpose : “If she asked you to remove her, then it’s fine, but if not, then you shouldn’t do that, as it’s bad practice.” “
                                                        • Another example: “404 means PAGE NOT FOUND (I hope you know that common knowledge).”

                                                        Your statements were pretty condescending to Ambarish and while I understand you were frustrated, he is too good an engineer to be talked to this way. I felt compelled to stand up for him.

                                                        At the end of the day, we got off on the wrong foot. This is the first time, in 4 years of running the service, where I have felt compelled to stand up for my team and just say that communication needs to remain polite and respectful. I am happy to continue the conversation over a call or over email, but I won’t be visiting this board to litigate this further.

                                                        Good luck and I hope you find a service that fits your needs.

                                                        1. 1

                                                          Yesterday, before I sent my comment in response your other comment, where you wrote answers to my questions, I replied here. It was a longer comment (not as long as last time, thought), where I was addressing your points. But it’s not here, not sure what went wrong, and I don’t have the willpower to recreate it. I’ll only recreate what I wrote at the very end of it.

                                                          I apologize for all the words I’ve written that you think were rude, inappropriate, or condescending. I’ll apologize Ambarish personally tomorrow.