1. 18

As per the usual …

Feel free to share what you’ve been working on here. Also mention if you need advice, help, or a second pair of eyes.

  1.  

  2. 23

    My entry is short this week as I’m not in the mood for talking much.

    If anyone knows of interesting compiler or related work (including, say, LLDB, LLVM, etc), I may well be interested to hear about it. I prefer working remotely but might be interested in discussing other options.

    As for Dylan, I don’t know what my future is with it at the moment. Multiple attempts to get others involved have largely failed. A comment that I made last week about what if I did something that mattered got 4 people to upvote it.

    There are so many interesting things that can be done on top of an industrial quality foundation. Instead, people quake at the thought of working on something big or involved. We’re in a time where people are proud to try to proclaim that they’re less than average, as a number of posters here on lobste.rs like to do. So I’m frustrated, I’m sad, and overall, I’m just really tired.

    I found an answer to my question last week. I started writing a new document as a project notebook listing existing projects, status, notes on them, etc. I just don’t see much point in continuing given the above and the lack of any payoff in doing so.

    1. 16

      Your Dylan posts have been really inspiring to me. Sometimes I worry that the work that I do is somewhat quixotic, or worse, aggressively counterproductive. Yours was neither, to me at least. You’ve been consistent, passionate, and downright interesting to read for the past few. Right now it seems like your project is just a struggling seed, but it’s probably gotten farther than the vast unseen majority of passion projects posted to message boards.

      I’m sorry that you’re feeling down. But if it means anything, your posts have brought me up.

      Also, for perspective. There’s like on the order of 3000 users on lobste.rs, I’m willing to bet that you have one of the highest average interest per post [avg comment score, large volume of posts]. So don’t sweat the upvotes. In our small pond, you definitely have our interest. For me at least, I strongly associate this small community with BruceM who loves OpenDylan. Be well, I wish you the best.

      Edit to update: TL;DR - I don’t know how I can help you with Dylan, but I want to point out that perhaps the payoff will be in an unexpected form; you inspired at least one person when they were down.

      1. 3

        Also, relying on others for payoff will not really lead to a lot of happiness, probably. (From experience, anyway.) Maybe just do things for the fun of it.

        1. 5

          To be sure, I do most of what I do because I enjoy it. I usually refer to myself as “fun-employed”. I do contract work that I enjoy and I do other projects that I enjoy (Dylan stuff being a part of that). My sadness and tiredness stem in part from some other issues (depression), but also the realization that I can’t achieve some of my goals for Dylan by myself, so I’ve tried hard to get others involved without much success. Maybe I let myself get a bit too deep into what I want out of Dylan for my own good.

      2. 6

        I am not surprised that it’s so difficult to attract new developers to OpenDylan. There are tons of new programming languages out there now and you’re competing for attention against companies like Microsoft, Apple, Google, and Mozilla.

        I hadn’t heard of Dylan before and one obvious question I have is: why should I care and invest time in learning about it? For example, Swift is obviously interesting if you’re working on iOS. Rust is a very interesting approach to systems programming. Go is starting to be everywhere so the big ecosystem around it makes it very interesting.

        What’s is special and unique about Dylan and where does it really shine at?

        1. 16

          Funny thing … Dylan was originally developed at Apple (along with CMU and Harlequin as partners) in the 1990s. It died at Apple for a variety of reasons, most of which had little to do with Dylan itself, much like many projects at Apple in the early to mid 1990s.

          What makes Dylan interesting?

          • It is a Lisp with an infix syntax.
          • Multiple dispatch / generic functions ala CLOS.
          • It has a Common Lisp style condition system.
          • It was the first or one of the first languages to offer a macro system with an infix syntax.
          • The core of the language is pretty elegant. The core concepts fit together cohesively and make sense together. (In contrast to some other languages around today.)
          • It has a pretty involved type system, including some aspects of dependent types, but without everything that one would want today. The type system is optional, but most commonly written code is typed.
          • It generates native executables and shared libraries.
          • Pretty good code generation as the compiler was previously a commercial product with a large team working on it who had years of experience implementing Common Lisp and other things.

          Personally, I came (back) to Dylan after a very unhappy time with seeing where things were headed with Scala, poor build tools, a slow compiler, a complicated language, slow development cycle, and decided that wasn’t a world that I wanted to live in. I also ended up not wanting to be tied to the JVM, which also changed my available options and interests.

          For someone who wants to learn and do interesting things, there’s a lot of open projects and interesting things to do.

          • Interested in numerics and generating good code and eliminating boxing overhead? We do okay at that, but can do better.
          • Ever wanted to hack on a compiler backend? We have a couple of those.
          • Been interested in designing aspects of a type system or working on implementing parts of a type system? We have a number of open projects in those areas.
          • Interested in FFI, integrating with Objective C libraries or other things? We have a number of open projects in that area as well, including the basics of an Objective C bridge and a fully implemented C-FFI.
          • Find macros interesting? We have some fun things to work on in that space as well?

          But those are things for people who are interested in working to extend their skills in the language design and implementation area.

          Without thinking too hard, we can probably come up with a good 30-50 projects, involving writing code that works with an industrial quality compiler implementation or related tools (like extensions to LLDB, etc) where the code involved could be in Dylan, C, C++, Python, JavaScript, etc depending on what exactly the project was. Some of these are more suitable for beginners, some for experts some for everyone in between. You don’t have to learn just by building toy compilers, at some point, diving into a full blown system will be a great experience.

          I personally enjoy writing prototype code in Dylan, but we’re missing libraries. Most of them aren’t too hard to do, but there’s plenty of help needed and plenty of areas where we can help someone learn wonderful and amazing new things.

          I’m currently building my own SAT solver as I’m interested in what might be possible with the addition of refinement types to the type system. (Among other potential uses for a SAT solver.)

        2. 4

          Sorry to hear this. You sound very tired.

          Can you put the Dylan stuff down for a bit? Just enough to recharge? FWIW, it sounds really cool; I’m just a bit committed to finishing something big of my own for once. I don’t think I’m the only one who’s in the “temporarily unavailable” category, either. I agree that people fear working on something big; our ‘technological plenty’ unintentionally creates a culture of consumption and taste over raw creation.

          I hope I can read more about Dylan via you and others.

          1. 2

            I think understand some of what you’re going through. At work, I’m involved with several huge scale projects, but sometimes it’s hard to get people to see past the immediate problems (JIRA-chasing, if you will) to see what we can do on a larger scale. It’s hard work, and an uphill battle, but if nothing else, I’m working to build the internet I want to see and quite frankly, I couldn’t do anything else. In the same vein, you’re building the language you want to see, and it’s an uphill battle, and I’m sure many times people don’t seem to see the bigger picture. Hang in there.

            1. 1

              There are so many interesting things that can be done on top of an industrial quality foundation. Instead, people quake at the thought of working on something big or involved. We’re in a time where people are proud to try to proclaim that they’re less than average, as a number of posters here on lobste.rs like to do.

              Are you being hyperbolic, or do you actually think that’s the issue?

              1. 1

                Both. It is certainly possible to pull out posts and comments saying similar things to what I mentioned. But should we always take them at their own word? It isn’t clear.

                This sort of comment isn’t all that uncommon: https://twitter.com/relrod6/status/513943840949288960

                1. 1

                  Yeah; it’s a standard boring comment. I wouldn’t worry about it.

            2. 8

              Last week, I got off my ass and penned a blog entry on Hython. Oh goodness, I could write a book about this if I wanted to. Submitted it to a few places, regretting it a bit later. I’m normally OK with most criticism, but middlebrow dismissals really irritate me for some reason. On the code side, I got started on implementing exception handling in Hython. I’m leveraging continuations to do it properly, but even with those there’s still a lot of thinking to do. I’m going slowly on this part and following a guide on doing it well. Hopefully I don’t have to circle back to it much after I’m done.

              This week, I’d like to continue working on exception handling, possibly while gathering notes to see what the next blog post on Hython will be about.

              1. 9

                Don’t let the negative comments get to you, honestly. Half of the people who make the snide comments are too lazy even read what you wrote, much less have the courage to write themselves. There’s a real lack of project-based Haskell writing, so blogging about these topics is quite helpful to a lot of people most of whom are silent.

                1. 5

                  Thanks for this writeup, I loved it. Looking forward to more.

                  Putting stuff on the internet is painful, but there are always more people that silently like it than commit uneducated non-constructive criticism.

                  1. 5

                    Why are you even reading comments on Reddit? :-) I loved the blog post! I’m also waiting to read more.

                  2. 7

                    This week, I plan to work on remaining PureScript compiler bugs for the 0.6 release, as well as getting binary distributions prepared for the three major OSs. I also hope to find some time to do a final review of my PureScript book, and to start thinking about printing paper copies.

                    Also, after Strange Loop, I was inspired to dig out my old Haskell miniKanren implementation and to try to fix some of its bugs. I might also make an attempt at porting the microKanren implementation from the paper.

                    At work, I hope to get my Haskell web service deployed and tested.

                    1. 3

                      I got asked by Will Byrd to help write a miniKanren in Idris and see if it’s useful at the type level, at all.

                      1. 3

                        I’d like to see that. Here’s my previous attempt to make a typed version. It’s messy and broken, and the new version will be tidier, using a sums-of-products approach. https://gist.github.com/paf31/7408295

                    2. 6

                      I was kinda sick last week, so this week is largely catch up for me. More and more Rust documentation, I have to finish off the new ‘30 minute intro’ and the new ownership guide, which is Rust’s most key concept, so it’s extremely important.

                      I’ve also been doing work to stabilize Rust’s SemVer implementation, which made it in last week. It’s not fully so yet, but it’s a candidate. Some functionality that belonged in SemVer was in Cargo, so last week I pulled it over into SemVer, today, time to bump that version and remove all the stuff I moved over. Fun!

                      I’m also looking for a new apartment this week, so that’s going to put a cramp in my non-$JOB contributions.

                      1. 6

                        I don’t know how it happened, but I’m writing another compiler to JS called frumpy. This one is implementing push-based first order FRP ala Elm. As an added bonus it has row polymorphism + first class records + curried functions. I think I’m becoming addicted to toy compilers.

                        My original project with linear types is temporarily on hold while I try to think of a nice DSL over llvm-general-ast. I’ve also heard that there might be a major revision brewing on that front so I’m a little leary of diving in only to rewrite the whole thing in 2 months or something.

                        I’ve also been reading a lot of type theory literature and have started a list of some good papers, suggestions welcome :) Papers I Love

                        1. 5

                          Rebar3. Implementing proper depsolving and update dependency for this thing is killing me. https://github.com/tsloughter/rebar

                          1. 1

                            What have you looked at for prior art?

                          2. 4

                            Last week I started writing another example app for Fire★ and decided that I needed vector clocks. So as of yesterday, Fire★ now has an implementation of vector clocks that is exposed for Apps to use so that you can write distributed apps that have some kind of eventual consistency of replicated data.

                            Now you can use vector clocks to determine if two edits to some data happened concurrently. Which allows you to decide if and how the data should be merged.

                            This week I am going to add and expose a text merge function for merging text, something like diff3 and updated the app editor to use it and vector clocks to make concurrent application programming much more user friendly.

                            My eventual goal is to have “smart” data types (concurrent data types) which combine vector clocks and data specific merge functions to automatically keep replicated data eventually consistent.

                            The goal is to make writing distributed apps with Fire★ even simpler than it already is.

                            1. 1

                              Just updated the built in code editor to use a new concurrent string implementation which combines vector clocks and strings and a merge function so that the editor can really work well with concurrent edits.

                              1. 2

                                I have done a lot of work on CRDTs, so know the background to auto-merging eventually-consistent state. I’m interested to know what algorithm you used for the merging?

                                Also, a quick note, you probably have implemented “Version Vectors”, which have the same structure as Vector Clocks, but different semantics. If you only increment the vector when the data changes, it’s a version vector; if you increment the vector when the data changes, and when you send or receive a message, then it’s a vector clock. Basho got the name wrong in Riak, and though we looked at changing it, it seems unlikely we’d be able to. For more on the difference, see this blog post by Carlos Baquero: Version Vectors are not Vector Clocks

                                1. 2

                                  You are right that the concurrent string implementation I made is version vectors. Thanks for clearing the terminology up.

                                  But the way I use them in the text editor is to send a message for all data changes, so the usage is vector clock.

                                  Fortunally I called the data struct “vclock” so I won’t have the naming issues riak had ;-)

                                  In regards to the merging, I am doing something simple at the moment. I use diff3 when I detect concurrent updates where the base string is also the current string.

                                  I am thinking a better approach might be to store last seen string from a node and use that. You have any papers that might help?

                                  1. 1

                                    So, the best place to start with CRDTs is here: Comprehensive study of Convergent and Commutative Replicated Data Types. This gives a broad overview as to most of the different kinds.

                                    Text we’ve found a hard problem, which is why riak sticks with allow_mult=true, then users resolve issues on their own. (Also because riak can’t know the semantics of the bit of binary data you inserted, it’s just bytes).

                                    A lot of research has been done on a system called “TreeDoc”. The research project, called ConcoRDanT has a list of publications, most of which should be open access. If you have any issues, I may be able to help.

                                    I’m also interested in your chat app. Do you use vector clocks in that? They’d be good for ordering messages by, so that causally connected messages always arrive in the right order (this is something Hipchat certainly doesn’t do, resulting in some messages that are out of order in weird ways). Ordering by timestamp almost works, but sometimes it doesn’t.

                                    1. 1

                                      Awesome, thank you for the references. Firts time heard about TreeDoc.

                                      This is what I love about lobsters!

                                      Also, chat app is definately next on my list.

                                      My goal is to eventually expose these data structures to app writers to simplify p2p app development.

                                      I already exposed the vector clock implementation.

                                      Combining version vectors and mostly automatic merging for base data types, and user configurable merging or version selection would be sexy for app writing.

                                      Text is hard. For now my basic assumption is that for pair programming, users would not be disconnected for too long. But u see your point that it would be good to allow user to select version by hand if does happen.

                                      1. 2

                                        So I think that for text merging, you should be able to detect non-overlapping edits and merge them successfully, but yeah, overlapping edits probably do require manual intervention. Though, once you’ve got manual intervention first, you can always add auto-edit-choosing later, given that it should be the easier of those two things.

                            2. 4

                              Distributed database to work in a mostly partitioned state to synchronize gif directories, https://github.com/ferd/figsync, based on a simpler database made on purpose for this thing (https://github.com/ferd/peeranha)

                              1. 3

                                I’m finishing up an instrumented, adaptive thread pool for the JVM: https://github.com/ztellman/dirigiste. I haven’t been able to find anything like it, which is surprising to me, given how mediocre the default offerings are.

                                1. 1

                                  How is the instrumentation accessed ? Can I query it via JMX? Is it controllable via JMX? How are governors implemented for clients that consume your library?

                                  1. 2

                                    The readme is pretty detailed w.r.t. how the instrumentation can be accessed and how the control loop works. No JMX hooks are provided, but you could easily propagate to it.

                                    1. 1

                                      Just went back and checked it out. That readme is a specimen in excellence.

                                  2. 1

                                    That’s really cool! I tried to do something similar in the past, but ended up building a terrible autopilot instead to learn PID control. Never came back to it unfortunately. Every other threadpool doesn’t do quite what I liked with retiring unneeded threads. I only wanted it to expand to provide for peak load and then follow it back down again. Watching with interest.

                                  3. 3

                                    This week I’m working on some initial discovery into a preforking background task queue for slow running jobs (our needs are video/image/audio encoding.) A crontab-style config will specify the max workers, user to run the job as, and the script to run. Each line in the config is a queue.

                                    1. 1

                                      Why not use Celery? Or Rabbit MQ and your client language of choice?

                                      1. 2

                                        Or Resque or Redis or AMQP or Gearman :)

                                        Being a personal project, I’d like to make it as suckless as possible. To me, this means simple infrastructure (no extra servers or apps necessary), built in portable ANSI C, and built in a Unix way.

                                        That said, if I was on a time crunch and needed to use existing technologies, I’d definitely use any of those mentioned above.

                                        1. 2

                                          In that case, maybe GNU Parallel is exactly what you are after.

                                    2. 3

                                      I’m working on writing provably constant-time crypto in idris. The idea is that bits can carry with them the number of times they’ve been operated on in their type, and then dependent types will allow a proof that whatever the inputs are, a constant number of operations have been performed, and thus the operation is (relatively) time constant. I have the whole operation-count-in-the type thing at least working (monads are super handy,) but I’m still working on getting anything proved.

                                      1. 3

                                        At the weekend I gave a talk on Pyland (a game to help children learn Python) at PyCon UK. We did a code sprint on Monday which seemed to go pretty well, hopefully we’ll have a successful build on OSX soon. I’ll be spending spare time this week trying to move Pyland forwards, particularly following up with teachers from the PyCon UK teacher track who were interested in helping.

                                        I’ve got to get back into PhD work this week and hopefully sort out the current set of results I’m working on. I’m still doing a whole bunch of extra reading related to lowRISC and our upcoming whitepaper.

                                        1. 2

                                          Catching up on an old repository, Jump-Location, which got a 10x boost in popularity last week after a big endorsement. It’s a smarter cd command for Windows PowerShell. I had forgotten about it since I changed jobs last spring and started working on a Mac, but the increase in popularity is demanding a lot of time now. I got a dev environment setup finally and am starting to chew through issues - though most of the work I’ve been doing is just reviewing and merging lots of pull requests. I hadn’t realized how large the community has become around it, but it’s fun to see all the discussion.

                                          1. 2

                                            I’ve just been mucking around with transducers; mainly reading this - http://conscientiousprogrammer.com/blog/2014/08/07/understanding-cloure-transducers-through-types/

                                            I should probably be doing other things, but haskell is a bit too distracting at the moment.

                                            1. 2

                                              First ever entry here so not sure if this fits in or not. Also its in C#.

                                              I have been working on a data generation tool that uses a syntax similar to regex (it differs in many ways). It started off as a library to generate flat csv files with various ranges of random data for each field and it has slowly evolved into a command line tool that I have added to over time. You can add generation patterns to template documents, use named patterns, add your own pattern files etc.

                                              https://github.com/SecretDeveloper/TestDataGenerator

                                              1. 1

                                                I’m building systems that allow you to dynamically modify your mobile apps on the fly. Right now I’m working on the web front end to control it but most stories usually includes web and iOS SDK code changes.

                                                1. 1

                                                  I’m just starting out on a research project to build a concurrency system for Idris. This will hopefully encompass a few things, including a better concurrency (verification) library in Idris (using Concurrency Calculi), a way to compile Idris into Erlang, and a way to reason about the behaviour of the Erlang Run-Time System, so that your Idris files can work with the guarantees it provides to provide verification of your concurrent programs.

                                                  The project will take a year (uni requirement), but I hope to have some interesting things done well before then.

                                                  Edit: I should point out there is already a Message Passing system built into Idris that Edwin coded up “for fun”. I believe programs work in it, but I’m not sure how much it is possible to verify them, and apparently there are so many locks that performance isn’t great. However, I did just find out how erts does its message passing, so that may be helpful.