-
I think the most underdocumented area in terms of computational complexity might actually be machine learning, deep learning and the like. I’ve never seen any talk about complexity there.
-
Great conclusion
-
I work in my spare time.
-
wsdookadr
edited
2 years ago
|
link
| on:
experience, thoughts and best practices for pdf server side?
I would pick one from these two options for server-side pdf generation:
using pdflatex (comes with the latex distribution called texlive in Ubuntu, in particular the texlive-base package)
using pandoc to convert a markup document to pdf (I think this uses pdflatex as well)
If you go with the first option, you may need to have some templates ready and fill them with data.
The second option is more convenient since writing markup (for example markdown) is easier.
-
wsdookadr
2 years ago
|
link
| on:
Poke holes in this idea - Distributed Search Engine ala block chain
unless I’m missing something YaCy comes close to what you’re describing
-
I looked into YaCy and Faroo before starting to think about this. Faroo is not even close. YaCy on the other hand is close enough but comes with a number drawback such as that it is very slow (due to fraud and spam protection) and does not rely on AI/ Learning but instead on conventional ranking methods. With that said it might be a good codebase or concept to start from.
YaCy also works conventionally by building an index and then traversing that index to respond to queries but what I have in mind (and have yet to start experimenting with) is No Index. Just a large scale global neural network that holds the information within that network. Now this comes with a million issues but on the other hand since you don’t fully understand the impact of your node on the search result (since is is part of the larger global network) you cannot in theory manipulate the search results..
-
PeARS is also worth looking into (recently funded by Mozilla).
About the NN, I’m not sure what that would do..
since you don’t fully understand the impact of your node on the search result you cannot in theory manipulate the search results
Not sure about how that would translate into practice.
YaCy on the other hand is close enough but comes with a number drawback such as that it is very slow
Reaching feature-parity with YaCy would require a ton of effort. But, you can always fork YaCy and try out your ideas and see how it goes.
-
-
-
wsdookadr
edited
2 years ago
|
link
| on:
What do you use to document requirements, specifications, and use-cases?
Yes, org-mode comes with any Emacs install. Since it’s plain-text, it can be versioned with Git (or any other version control system). Org documents can be exported to a variety of other different formats including Word (via org->odt->doc/docx) and PDF (via org->tex->pdf).
Orgmode features code-blocks (the equivalent of reStructuredText’s code directives ). These are extremely useful since they can be run inside the document, and their result can be included in the document, either in the form of syntax highlighted source code or the results of that source code (data, images, or anything else that the code block generates) or both. For the code blocks, a wide variety of languages are supported (some of those languages are DSLs for drawing diagrams, sketches, for example you can design simple UI mockups via plantuml code blocks).
So you can include diagrams about systems, UML use-cases, it also has tables (fully equipped with excel-like formulas), you may also use state-diagrams to describe the test procedures you mentioned.
If you have larger documents to write, you may also benefit from
#+INCLUDEdirectives that org-mode provides you with. This helps separate your document into sections/chapters.Other use-cases include:
keeping track of upcoming events through the use of a built-in agenda
Having said this, Word and Excel remain the #1 word processor and spreadsheet programs out there. Orgmode, as complete as it may be, is tailored for technical people who can take advantage of its features.
-
wsdookadr
2 years ago
|
link
| on:
Engineers Shouldn’t Write ETL: A Guide to Building a High Functioning Data Science Department
excellent article, I wish there were more of these articles around for different types of teams and the composition and interactions between roles
-
because choosing your employees/teammates based on 20 lines of python is definitely a good idea.
-
the blog post starts from a problem and describes a solution to that particular problem. sure, other people want to optimize for something else, everyone can, they are free to do that. if you believe a different problem should be solved, that’s interesting to me, and I look forward to reading your problem and your solution to it.
-
It is not much of an exaggeration to say that trying to find algorithmic or technical solutions to humane and social problems is one of the biggest risks to humanity’s mid-term future.
Someone else said it better: ‘any sufficiently complicated technical problem is a political problem first’.
This for sure is a nice exercise in algorithms for matching purposes and for automation, but dealing with humans with a blunt algorithm is a terrible mistake, you must approach human decisions with human sensitivities.
-
-
-
One nice feature of FIFO’s is that if you write less than
PIPE_BUFbytes, the write is atomic. This is not necessarily so useful in the 1 producer multiple consumer case, but in multiple producers one consumer case it means the file lock can be removed alltogether. Unfortunately,PIPE_BUFis small on OS’s like FreeBSD (512 bytes, I think), but Linux is around 8kb, I think.The other thing to note is that the FIFO is not doing fan-out. Each consumer is destructive to the FIFO.
All-in-all, I think FIFOs are an underused tool in a lot of software. The kernel is pretty efficient at shuffling data around, the cost is a a syscall to read/write to a FIFO, but it’s a nice way to connect components and maintain process boundaries. Unless someone is doing really high throughput/low latency work where disrupting the CPU cache is a serious problem, using FIFOs to connect components will probably have no negative effect on performance.
-
-
btw, FB has also announced a Parse-compatible API server based on nodejs and express (BSD license). It seems that users are happy with this one as well
-
Going to spend some time looking at self hosted continuous integration solutions. I’m unhappy with jenkins and not very interested in moving to one of the hosted ones out there.
-
-
This is an oath about things 99% of programmers have 0 control of.
If this oath is meant to make us all feel bad about the current state of software “engineering”, it does a good job.
-
That doesn’t seem true to me at all. These, especially, seem like things I’ve tried to stick to that others ignore:
1 & 2 are simple: do no harm, insert no back doors.
5 - Don’t take shortcuts that don’t agree with structure of existing systems.
6 - Team productivity is more important than your personal productivity. It’s fine to just-get-shit-done, but not at the cost of other’s ability to do their work.
8 - Others rely on your estimates. If you’re not sure how something will work, simply don’t make an estimate of how long it’ll take to code it up.
-
-
-
Maybe this oath was inspired by this article (the article briefly compares the construction industry with the software industry and how roles are viewed and the meaning of the term “engineer” in both of those industries) . This oath can cause some confusion. Maybe an analogy would help: a construction worker is a subordinate of a construction engineer the same way as a software engineer is the subordinate of a manager/teamlead. The programmer is at the bottom of the hierarchy just as a construction worker is at the bottom of the hierarchy. Isn’t subscribing to such an oath something that a manager should do and not a software engineer ?
Also, pretty much every employment document out there would force you to comply to similar requirements as this oath. So.. it’s a bit confusing why a public oath like that is necessary, maybe just as template?
But.. there’s a confusion even in the title of the oath. What is a programmer ? What is the difference between a software engineer, a programmer and a developer ? I’ve read various answers on the difference between those terms and there wasn’t any consistent definition.
Java often includes the algorithmic complexity, e.g. https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html
Yeah, but mostly for their core stuff.
I missed that constant-time performance comment - which is very helpful, but wish it was called out a little more explicitly. I wish the memory/runtime complexity were called out as explicitly as the load factor that is liberally sprinkled throughout.