[Comment removed by author]
Interesting idea! I hope you share your results.
Not really a fan of this idea. This article isn’t so much about “defending” your website as it is about attacking anyone who scans it. Vulnerability scanners are often run from servers that are themselves compromised, so retaliatory attacks like this can further victimize people who have already been owned :(
Still pretty neat on a technical level though.
Many people are not even aware that they’ve been compromised… At least that helps in a way!
Just because you’re being attacked from compromised server, doesn’t mean that you’re not being attacked.
I’ve been running SSH on multiple servers with non-standard ports for years. Yet, I rarely, if ever, get failed login attempts. Is this really a thing?
Worked for me for a long time, but they’ve found me now :’(
This is neat. I really enjoy seeing how short the experts can make one-liners.
When I put F5Bot public, the Reddit scraper was only a handful of lines. It worked for a while, but today it’s thousands of lines. It’s amazing how many edge-cases come up to the surface. Also, the Reddit API is really quirky.
I’m trying to finalize the scripting interface for Tulip Charts. I really want to release a public alpha soon. I’m trying to find the right balance between brevity and simplicity and elegance fro the API. In the end I guess I’ll just need to pick something and go with it.
Why wait to open source it if that was your intention from the start?
No real reason, other than it’s less work to publish it later. I’ll probably put it up on Github soon anyway.
I’m still plugging along on the business end of Turnkey Telemetry.
In my spare time, I’m trying to make F5Bot monitor all of Reddit, instead of only specific sub-reddits. Reddit’s API and rate limiting don’t make this very easy. I think I’ll need to take an approach that makes multiple simultaneous requests.
I’m trying to find better sales channels for my startup, Turnkey Telemetry. It’s not as fun as writing code or building hardware. I could use some advice if anyone has experience with the sales end of a similar product line.
It looks like https://barnacl.es/ is the place for you.
Hadn’t seen that. Thanks!
Looks like a fairly complete list of examples.
In practice, the problem I see most often is the implicit pointer cast. Implicit casts are an inherent part of C programming, so it actually shows up everywhere. However, C++ culture holds that casts are generally evil, and the language makes it difficult (while preferring C++’s flavor of OO instead). Most of these other issues you don’t often see in real code.
Also, I find it a bit odd to phrase it as C being a subset of C++, instead of saying that C++ is a superset of C. I guess they technically mean the same thing, but saying C is (not) a subset almost implies that C came after C++.
I got these from a couple sources, then someone pointed out this Wikipedia entry which looks more complete.
Yeah, the implicit cast is part of the C philosophy that the programmer knows what they’re doing.
I phrased it that way because that’s how I’ve been hearing the claim, C is a subset.
This is cool. What algorithm are you using?
On another discussion their response to a similar question was something like this:
I can share that our platform is built on top of ARIMA models, but with a lot of pre-processing work done previously to try and figure out automatically the best parameters to use, as well as a lot of previous hand-tweaking done by ourselves in-house using different datasets (we started out tuning it for forecasting energy consumption, but figured that the resulting models were performing well enough to warrant testing in other domains).
Trying to get the last few bugs out of F5Bot, my free social network monitoring service. I think I’m going to rewrite a lot of the parser code today.
Could use a forgot password function.
Yeah. I’m working on that. I’ve had a couple users email me, and I’ve manually reset their passwords. If you need me to manually reset yours, just let me know.
I made something similar too, but it sends emails instead of Slack comments. It’s a free service called F5Bot.
What did you use to build this? (if I may ask)
Because I wanted to put this out there for free, that lead me to chose the most boring and simple stack I could think of. I maintain a pretty large project in Node.js, and it’s great for what it is, but I can get a PHP site up and going with much less hassle.
I take it you use a cron job to trigger website monitoring then? There’s a lot to be said for going with something as boring/quick as possible.
Yep. It’s a cron job the runs wget to load the scraper page (which includes some security checks first, of course).
Lovely little product that I would pay for, the trackreddit interface is abysmal.
Tech wise though - is the reason you don’t support searching the entirety of Reddit a scaling issue? I ask because the global comment firehose seems to be freely available, so I’m wondering if there is a reason you don’t use that over selecting specific subreddits. Thanks!
Thanks for the feedback!
I built F5Bot in only a couple hours, so I didn’t do a lot of research or planning. It appears that the ‘comment firehose’ you posted doesn’t even go back one second in time. Is that right? I’m not sure how I could realistically use that without hitting Reddit several times a second, and even then I would miss a lot when Reddit goes down (which is often, I now know). Also, when I posted to Hacker News, a couple commenters mentioned Reddit API limits.
So I just took the lazy/easy way by monitoring only the few subreddits that I actually cared about anyway. In this case, I can check back every couple minutes, and if sometime goes wrong I can check back even later without missing anything.
Would this be substantially more useful to you if it pulled all of Reddit, instead of the current subset?
It’s nice of you to say I could charge money for this. I think I’ll leave the current feature-set up for free. Realistically, it only took me a couple hours to build, so I don’t think it makes sense to monetize that. In fact, I was thinking about going the opposite way and open-sourcing it. I might add premium features in the future, but I’m not sure yet.
Hey, we’ve documented some API rules here.
You should use a unique user-agent as mentioned in the previous link. Also using OAuth will increase the rate limit to 1 qps. If you’re using PRAW, it’ll automatically handle rate limiting for you. If you have any more questions, feel free to reply here, post on r/redditdev, or PM me.
Yes. Social Media monitoring is quite a large market: Mention, Hootsuite, etc but they all come with complex UI, reporting and cost quite a bit. There are a few in the cheaper “just notify me” space but the ones I have used have been fairly awful UI wise.
As to hitting reddit every second, I had made the assumption you’d be doing that anyway - but yes, it would be worth reading around to find out what they deem acceptable.
Just checked out trackreddit. You weren’t kidding about the interface! I guess maybe they were thinking to make it more powerful, but man, they did not optimize for the common use case at all.
It actually works, but the email notifications send you to their mobile feed which has an ever worse UI than the main app. Also when it came to cancel, you can’t within the app and they ignored my request for 2 months leaving me with no choice but to raise a PayPal complaint.
Felt like a hobby project gone wrong.
That’s a feature on Lobsters already. I don’t know if it requires them yo use the @ before the name, though. They always fid that when I gog notifications.
I didn’t know about that.
F5Bot allows you to monitor arbitrary keywords or phrases. So you can get notified whenever someone mentions your startup, or even your competitor.
That would be an advantage over the built-in functionality.
Thanks for posting this. I’ve used vim as my primary editor for a very long time, and I learned a few things here.
When building F5Bot over the last couple days, I waffled on which framework to use. In the end, I decided to go with nothing but my own css, and I’m pretty happy with the choice. It actually felt really liberating, and the page loads are noticeably faster than with something like Bootstrap.
I think that limiting yourself to only a few lines of css is extreme, but I do think that I’ll be defaulting to no framework in the future.
Oh I agree on CSS. I personally use quite a bit more than what’s in this post (my style.css) and don’t have any real purism about it. But I wanted to point out that the required minimum to get working semantic HTML that renders ok on the modern web is very reasonable and doesn’t require becoming a massive web guru. (I started out with something pretty close to this, and only learned some more CSS later.)
Many of my colleagues in academia used to maintain their own pages in plain HTML once upon a time (the classic example.edu/~username/ sites that lived in your ~/public_html directory on the departmental Unix server), but I get the impression that for many people who don’t “do” webdev, the modern web is seen as too complicated to DIY, so you need to go directly for a CMS or a framework unless you want to invest a bunch of time learning the ins and outs of the HTML/CSS/JS stack. I mean, there are small static sites with 5 pages that are built on Wordpress, because that’s just the default thing people reach for to make a site. Which is understandable, since I almost had that reaction myself: I load my old site on a mobile phone for the first time, notice it renders horribly, Google to try to figure out what’s going wrong, then end up down a rabbit hole of blog posts about “responsive design” and “grid models” that are way more involved than I wanted the answer to be.
I eventually stumbled across the four minimal things I suggest here from various sources, e.g. I think I first got the viewport line from a Google Webmaster Tools help page, realized I needed a charset line the first time I tried to use an m-dash in a document and it broke, etc. I hadn’t seen them summarized anywhere (along with an explanation of why things like a viewport where width=device-width aren’t the default in the first place), so figured I’d write it up. The other part of the motivation for writing the post is that I’m not even sure if these are the minimal set; since it’s picked up entirely ad-hoc from fixing things, there might be some 5th thing that has a good complexity:advantage ratio that I should know about.
Agreed. I didn’t mean to come off as critical in my first comment. I think your write-up is a very good one, and you’ve identified important declarations that I hadn’t really thought about before.
Like many developers, I didn’t really bother to learn css thoroughly. Css didn’t exists when I first learned HTML. Once I started making web pages seriously, I just copied and pasted Boostrap examples like everyone else.
I think it can be really intimidating for some developers to forgo the heavy-duty framework. Few people understand how the frameworks work exactly, because they are quite complicated with many moving parts. And I thought that if I couldn’t duplicate Bootstrap, then I was stuck with it. Of course, it turns out, for most things you just don’t need all that complexity, but that was hard for me to realize.
I think it was (warning profanity) this website and this website that finally convinced me to throw away the 2mb framework and start from scratch.
What convinced me to dig into it a bit more personally was a very bad experience with Wordpress (plus a few less-bad-but-still-annoying experiences with Wordpress). Wordpress “just works” to set up a simple website initially, but if anything goes wrong with it, good luck…
I tend to use straight css that I pass through cssnext with postcss-cli. It takes care of alot of cross browser conpatibility.
I’m putting the finishing touches on my weekend project: F5Bot. It’s a real simple service that monitors Lobsters, Hacker News, and Reddit for keywords. It emails you when it finds something. I’m just working on making the error handling and logging a bit more robust now.
I’m thinking about updating my math expression evaluation library to allow arbitrary operators to be added at run-time (with custom precedence and associativity rules).
Working on Tulip Charts. Hope to have it ready to release an alpha very soon.