I’m one of the creators of Sourcegraph. We released this master plan to explain what we’re up to and why it matters.
Make basic code intelligence ubiquitous
Make code review continuous & intelligent
Increase the amount & quality of open-source code
Happy to answer any questions here, and I’d love to hear what people think. We know it’s a lot of work, but this is inevitable and will make development smoother for everyone.
Just released a big update to the Sourcegraph Chrome extension that adds support for annotating GitHub PRs and diffs with jump-to-def links. This was by far the #1 user request. Direct link: https://chrome.google.com/webstore/detail/sourcegraph-for-github/dgjhfomjieaadpoljlnidmbgkdffpack.
The post a few weeks ago https://lobste.rs/s/xljhkx/browse_github_like_ide_jump_def_doc was for the initial release, to add IDE-like features when browsing files (not PRs) on GitHub.
Do you still only support java / go?
Yes, we currently support Java and Go but new languages are coming very soon.
I don’t know that I would call this automatic pair programming. It looks cool and sounds useful, but most of the time I’ve spent pairing hasn’t been looking up documentation while someone else writes the actual code or vice versa.
Outside of actual utility or quibbling over the title, I would be concerned about using this because it is leaking a lot of stuff to an external service.
Yeah, we tried to make it send the absolute minimum amount of data to an external service. It doesn’t send the code files, only the resolved tokens' package (Go import path), repo name, and definition path (e.g. “FuncName” or “TypeName/MethodName”). More info in the editor plugin readmes at https://github.com/sourcegraph/sourcegraph-sublime and https://github.com/sourcegraph/sourcegraph-vim. Some folks indeed won’t be able to use it because of their company’s policy, so we’re just trying to do the best we can to be transparent and conservative here.
This looks really cool and is one feature that I think is missing from Dash. There are tons of integrations for editors with Dash and its a great resource to be able to quickly look up documentation when you need it but its definitely missing a curated set of examples depending on the documentation you’re looking at, i’ve found most documentation that I use at least is pretty void of usage examples. This means generally if i want to find how its used I’ll need to go over to Google to search for actual usages.
I like the way Dash does the integration better than having an always on window that updates while I type, personally I would get really distracted seeing the right side of my screen always flickering with new information.
Sourcegrapher here. Thanks for the kind words. We built this editor integration because we 100% agree with you—actual usage examples are super valuable.
You can turn off the live-updating, always-on behavior in the editor plugin. See https://github.com/sourcegraph/sourcegraph-vim#vimrc or https://github.com/sourcegraph/sourcegraph-sublime#auto. Then you just use a hotkey to jump to usage examples. Or you can just keep that browser window in the background (that’s how I use it, since I use a full-screen WM on Linux).
(One of the creators of Fair Source here…) Fair Source is not an open-source license. It is intended to be an improvement over closed source (GitHub, Bitbucket, etc.) and open core (where often the important bits are closed source).
improvement over closed source
Improvement by what measure? From the page:
Releasing a product’s source code makes it more valuable to customers by enhancing extensibility and building trust.
I don’t see how those properties are enhanced by this kind of license. Extensibility is provided by good modularization and useful APIs and well-defined plugin points (along with good documentation). Trust is built via positive exchanges over time, it’s not instantly provided because I could (though won’t) comb through the code.
Sourcegraph CEO here. We have large enterprise customers using Sourcegraph and are excited to release it publicly (with source code) so anyone can start using Sourcegraph for their team’s code.
Feedback is much appreciated!
More info at the announcement blog post and at Sourcegraph.com.
Looks neat, I like the features. I often find myself browsing github for something, then having to jump back to IntelliJ so I can follow method calls and “find usage”.
How does the “inline discussion” stuff work once the code has completely mutated past what it once looked like? Is there some kind of timeout that makes these notes fade over time?
This might be a strange question, but is it possible to register via github sign-in? The biggest benefit (imho) to github for OSS projects is the network effect: anyone can stop by and send a PR, file a bug, etc. Having your own tracker definitely brings tech benefits, but the downside is that there is more friction to getting outside people to contribute. Or perhaps this is really targeted more for internal development that isn’t necessarily OSS?
Probably just my connection (DSL yay), but many page loads are super slow for me. 3-14 seconds in many cases, and it looks like the json XHR RPC calls are doing a lot of it (long load but only a few bytes).
Any chance of getting Rust support? :D
EDIT: srclib looks neat! Will have to poke around that a bit more :)
Thanks for checking it out! We’ve resolved some perf issues and the site should be faster now. If it’s still slow, can you send me the page URLs that are slow (email@example.com)?
Inline code discussions are currently tied to a line range in a file at a specific version, which makes them useful as a jumping off point to filing an issue in Sourcegraph Tracker. In the near future, we’d like to make them attachable to specific pieces of code (e.g., a function definition), as well, and in this case, the comment can remain as a relevant contextual item for the duration of the lifetime of the function. I like the idea of fading discussions out over time, as well.
Rust is on the roadmap, but we welcome any and all contributors to srclib, our open-source core code analysis library :)
We don’t support GitHub sign-in currently, but that’s something we’ll consider in the future.
It’s a great announcement!
Hey, thanks for checking out Sourcegraph! Apologies for the slowness on the site yesterday. We’ve been working through some performance issues and it should be much faster now.
We use server-side rendering for the mostly static parts of the site – good for SEO, performance, etc. For the more interactive parts (code browser, etc.), we use React to make more dynamic features like the code browser pop up box and smooth jump-to-def within a file.
This looks really nice. Particularly once there is more language support it seems like it’ll be better than the other tools I’ve used. I did notice a bug, in Safari (9.0.1 on OS X 10.10) the following link appears to have a redirect loop:
Edit: Beyond Go & Java I’m particularly interested in Python (painful to analyze, I know). The other items on my wish-list would be Protobuf (if Sourcegraph could understand the correspondence between the generated code and the original file), Swift, and Objective-C. In an ideal world you’d be to jump-to-definition across RPC boundaries (say iOS client into Go server code) but there’s probably no sufficiently general way to do it to make it worth doing.
I don’t think the findings are that surprising for anyone who uses Go, although no one guessing that len was the most used is surprising.
I think the fact that nearly all of the top 150 are standard library functions is a good indicator of the breadth and quality of Go’s standard library. There are very few cases where you’d go out and find an alternative, and if you do, it’s usually for increased performance at the cost of features.
Still disappointed to see io/ioutil.ReadAll halfway down the list. ;)
Yeah, agreed. (I’m the original author.) It will be interesting to compare stdlib usage among various languages.
This is amazing, but I’d really want it to work for Java. Trying to navigate through 8 layers of package directories makes it really difficult.
You can use https://github.com/buunguyen/octotree to simplify the navigation. But it doesn’t seem to work with the Sourcegraph extension, or vice versa, yet though.
Java support is next for us. If you +1 or follow the issue at https://github.com/sourcegraph/sourcegraph.com/issues/172, you’ll get an update when it’s ready.
I had the same idea. Really cool project, but I would love to see this for more languages (C, C++, Java). Arguably, these languages should be easier, because of explicit, static typing and a strong symbol hierarchy.
I am the author and happy to answer any questions anyone has here!
Why did you use globals for state? Isn’t it just as easy, and less dangerous, to close over state in handlers, e.g. like this?
Our tests run in serial and use mocks, so there is currently no gain to us from having handler context.
I was expecting to get something back for sys.exit as it mentions python but I don’t get any results back.
We just pushed an improvement to make it get Python C definitions better (of which sys.exit is one). Expect a blog post announcement soon.
Sourcegraph co-creator here. Thanks for posting this! We pushed a bunch of improvements to Sourcegraph to make it faster and easier for you all to search for code and find real usage examples (taken from other projects on GitHub). We’re developers, too, and we want Sourcegraph to be the developer reference tool we want to use 100x a day. Let me know if you have any feedback. :)
It’s looking swankier and swankier every time I check it, great work!
I noticed the golang crawling has some bugs in it though:
In the side nav, the dependents count is often incorrect, for example: https://sourcegraph.com/github.com/go-yaml/yaml/.dependents
Traversing dependencies links are sometimes broken, for example: https://sourcegraph.com/github.com/go-yaml/yaml/.dependencies -> https://sourcegraph.com/gopkg.in/yaml.v1
Searching for gopkg.in modules does not work, for example: gopkg.in/yaml.v1
gopkg.in-wrapped dependencies are not recognized (that may be asking for a lot, but would be very handy, especially if they got merged under the parent package unlike http://godoc.org/gopkg.in/yaml.v1?importers which keeps them separate.)
In general, the results seem to be quite different from godoc.org, for example: http://godoc.org/github.com/go-yaml/yaml?importers
From the scenarios where Sourcegraph did work, I found it very handy. If most of the above was fixed, I can see myself growing fairly dependent on the service. :)
Thanks, this feedback is super useful!
I filed bugs on us for points 1-4: https://github.com/sourcegraph/sourcegraph.com/issues/161 and https://github.com/sourcegraph/sourcegraph.com/issues/162.
As for having different results from godoc.org, yeah, there are a lot of improvements we could make there. In the case of that link you posted, it looks like both Sourcegraph and godoc.org each had 1 importer/dependent but they were different. I just triggered a build of the one godoc was showing, and now Sourcegraph has both. Of course, in many cases Sourcegraph will be worse than godoc.org–but we’re working really hard to improve it all the time.
Please keep the feedback coming!
Looks very cool. We’ve released GraphKit, a set of open source tools for source code analysis at https://github.com/sourcegraph/graphkit. It supports fewer languages but performs deeper analysis of their source. (We use GraphKit to build https://sourcegraph.com.)
I’m working on Sourcegraph, code search for open source. It provides good usage examples and docs for you when you’re coding. This week, I’m focused on improving the snappiness of search and fixing outstanding errors that have popped up (let me know if you find any more!)
Nice project! I like how methods are linked in the source code to their method pages.
What search backend are you using?
Sourcegraph uses a combination of PostgreSQL and Elasticsearch for searching. We’ll write up a tech stack post one of these days, but ask away if you have any other questions.
I finished up an ERb-style templating language for Go called ego. It transpiles to pure Go so it allows for any Go constructs, statically compiles, and supports compile-time type checking. Feedback welcome!
I’m using that inside my open source Mixpanel implementation that I’m currently calling Skybox.
I saw this yesterday and thought it looked interesting. What made you decide to not use Go’s text/template and html/template? Have you found that your ego templates are overall more or less complex than the equivalent stdlib templates?
I’ve used the standard library templates before and they work fine but I run into a few issues:
I always have to reference the docs a lot to remember how to use the pipeline syntax.
I have to register any functions I want to use in my view.
I have to separately run go-bindata to embed templates into the binary.
There’s no static type checking.
Nested templates are a pain to manage IMO. With ego, everything is just a function call so they nest well.
So far it’s been pretty straightfoward. I don’t have to context switch in my head between Go syntax and template syntax. It’s all Go code. Compiling templates using the ego CLI is quick and painless too. I’m using line pragmas in the generated source so template errors are reported based on the template’s line numbers (and not the generated code’s line numbers).
Cool! The type safety sounds particularly nice. I look forward to trying out ego!