1. 1

    Not sure what I thought this would be, but it’s pretty cool, tbh. I’m a big SQLite fan, and can see this being helpful for a specific kind of project.

    1. 1

      I’m always here for more MLOps and DataOps talk

      1. 2

        Thanks for sharing that! As someone who’s primarily used Windows for years due to work reasons (large enterprise orgs like that OS and I was a .NET developer for years), I had to avoid languages that didn’t run well there, like Python and Ruby.

        Now I’m a data engineer and do a lot of work in Python – tools like Jupyter have made that a lot easier to be portable when I’m not on a client that either gives me a Mac or Linux machine or creates a cloud based environment I can remote into. Sure, there’s a holdup when they are weird about giving admin access, but for the most part, doing things on Windows just works for me.

        This is awesome and allows more folks like me to break in to that market.

        1. 5

          Writing a conference talk that I’m supposed to give next week.

          I, uhhh, procrastinated a bit. :(

          1. 2

            Good luck :)

            1. 1

              Thanks!

            2. 2

              You’re starting one week ahead? That’s like crazy early! :o)

              (I’m notorious for pulling all nighters to get them ready in time)

              1. 3

                Don’t we all?

                1. 2

                  It takes a while to make some sick memes with pictures I can 100% say I either own or are totally free for use. ;)

                  1. 2

                    Can confirm I still was finishing up slides and rehearsing an hour before. ;)

                1. 14

                  Oh wow, I had no idea, and as someone who’s begrudgingly worked with Oracle’s products over the years, this isn’t shocking. If anything, it explains a lot.

                  They’re somewhere between a law firm that sells some software and a drug pusher, and their products are pure torture to use. Of course the CIA was involved.

                    1. 2

                      Meaning…?

                      1. 5

                        It probably counts for his/the companies assholeness.
                        I was going to say he’s into objectivism, but that’s not the case.
                        To satirise it

                        1. 1

                          Not to defend a philosophy I don’t entirely agree with, beyond pointing out that my friends who are objectivists are some of the nicest people, I have to ask if you ever made a strawman out of a generalization of sample bias? And backed it up with satire that’s practically a parody of itself?

                          I mean, giving objectivists shit because some large business owners read Atlas Shrugged, and apparently understood it, despite being douches, is kind of old.

                          Nice bit of trivia, but ultimately speculative and irrelevant.

                          Feel free to reply you never meant a generalization or something.

                      2. 2

                        And that explains everything else bad about Oracle.

                    1. 4

                      I mean, this was interesting, but I seriously thought it was going to be about Frinkiac, the meme finder for Simpsons GIFs, or Professor Frink himself.

                      Also, +1000 for the Zardoz reference.

                      1. 6

                        That’s actually the source of the name! From the site:

                        Frink was named after one of my personal heroes, and great scientists of our time, the brilliant Professor John Frink. Professor Frink noted, decades ago:

                        “I predict that within 100 years, computers will be twice as powerful, ten thousand times larger, and so expensive that only the five richest kings of Europe will own them.”

                      1. 3

                        I flagged as off-topic because there is no technical content in the video and the content is anecdotal reflections on social issues at best.

                        1. 3

                          I think that’s the point. For the person doing the video, and people like him, they feel like they CAN’T level up on technical skills until they prove themselves in multiple other ways, often involving issues outside of work.

                          This has been a longstanding thing with all areas of trying to increase diversity in our industry. Conway’s Law states that systems mirror the organizations that created them – this means that if your organization is exclusionary (even unintentionally), so will the things your organization creates.

                          Look up the term “code-switching” to get a better idea of what he’s referring to for part of this video.

                            1. 2

                              Why?

                            2. 1

                              I’m disappointed.

                              1. 1

                                I think that’s the point. For the person doing the video, and people like him, they feel like they CAN’T level up on technical skills until they prove themselves in multiple other ways, often involving issues outside of work.

                                This has been a longstanding thing with all areas of trying to increase diversity in our industry. Conway’s Law states that systems mirror the organizations that created them – this means that if your organization is exclusionary (even unintentionally), so will the things your organization creates.

                                Look up the term “code-switching” to get a better idea of what he’s referring to for part of this video.

                              1. 7

                                Barely. I’m barely holding up. My work has doubled down on a number of things (my employer is beyond great, it’s the client that seems to have not let up despite working in a crisis.) The only saving graces are my general privilege, the fact that my wife wasn’t currently working anyway when preschool closed, and that preschool reopened a couple weeks ago. The last part helped dramatically because now my wife doesn’t need me to timeshift nearly as much to do parent duty on days where she needed to brave the grocery store, was exhausted, etc. It also helped our daughter’s general mental well being and stress levels (partially because ours went down).

                                I’m still doing relatively well profesionally despite all this, but that’s because I was very selective about what I did (I dropped 90% of anything extracurricular and gotten a whole team to help with what I was doing alone at work). I went through with speaking at a virtual version of a conference I was supposed to do last month, because I am making the most of my 10% of extracurricular to do double or triple duty.

                                Beyond that, forcing myself to take breaks from people in general (my work has been very collaborative lately so between that and family, I get no reset time), from work during the day when I can to be with family, and from both (albeit via video chat) on regularly scheduled occasions weekly with a small group of friends. It’s helping, but still having a lot of trouble and as an American, the general state of things adds more anxiety (mostly because I feel too burnt out to do more, but also because I feel like that’s not a real excuse given that I still have it relatively easy).

                                1. 2

                                  Two days of planning out the next twelve weeks of work for my job, despite being on a scrum framework.

                                  If your workplace ever says they’re going to SaFE, run like mad. Burn the place on your way out.

                                  SaFE isn’t safe for your psychological safety.

                                  1. 6

                                    Are you trying to switch technical specialties or coming in new? I’m assuming the former from your ask, but going to generalize for all readers.

                                    If it’s the latter, I’d say most data engineers seem to come from either a standard development background focused mostly on back end work. You know how to do ORMs but are also decent at SQL and database structure, but most importantly, have been doing coding for a while.

                                    OR

                                    You come from an analytical background, data analysis, data science, visualization, etc. You may not be as proficient at production coding practices, but you really grok data and all its messiness, when to do an aggregated, denormalized structure vs a normalized transactional one, etc..

                                    Either way, you need to enhance whatever skills your lacking on either the standard code side (likely in Scala or Python) or you need to get a little more familiar with various data storage types (relational databases, graph, document store, etc.) and the weirdness inherent in data that analysts and statisticians are happy to tell you about.

                                    If you’re coming in totally cold out of a boot camp or college, then the basics are really try to understand at least the basics of SQL and how databases work (this is the basis of all the other more complicated stuff anyway), a programming language (Python is your faster bet here if you’ve never programmed), and a sampling of a few other things – data modeling, how APIs work, ORMs, what data scientists/analysts/visualization people do, how back end datasources work for standard software, a bit of DevOps, etc. The big parts are the basics of data and programming to access and process it then store it elsewhere though because that’s the foundation.

                                    I’d go into more detail, but really, there’s so many database types, orchestration tools, frameworks, cloud services, etc. that it would make the Javascript folks heads spin to know they all exist. The rest of it is really more about the type of work you’ll end up doing, and being flexible is the best way to go about it.

                                    Happy to answer more questions since I sort of fell into it over the years, but work with a variety of other data engineers and also make a point of teaching and spreading the word when I can.

                                    1. 2

                                      Thank you for writing such a thoughtful reply and It means a lot. I have mostly programmed in python and I do have the knowledge of SQL but not advance just beginner since I mostly did ORM instead of writing plain SQL queries but I would love to know more like what resource can I consume since to broaden my knowledge

                                      1. 1

                                        No worries. The main reason for learning SQL is it’s the basis of other query languages for non-relational databases. I’d say knowing the basics will get you a ways from that standpoint, and the advanced stuff you end up picking up as you do it.

                                        I stumbled a lot into how some of this works by experience, but I’d say doing basic free tutorials for various database types (Postgres for relational, MongoDB for document, Neo4j for graph) are a good start. There’s a ton of links out there on Dev.to and various Medium sites (Toward Data Science often has some good stuff) to get you on more of the basics of data engineering.

                                        This is also an interesting way to look at things since you’re already doing Python.

                                        https://medium.com/@maximebeauchemin/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a

                                        1. 1

                                          Beyond that, I’d say look into the idea of DataOps (using DevOps, Agile, and Lean principles to work data projects) for some good way to get up to speed on the up and coming way to go. The DataOps Podcast is a great way to do that, along with resources that are put out by a few companies pushing the idea forward that have put out a ton of free info.

                                          Also, if you haven’t found it already, the Data Engineering podcast is a good place to get an idea of general toolsets and challenges being used.

                                          Probably a lot more to cover there – I’d say just getting the basics and a good hold on how to work with data in general as well as make sure your understanding of different data storage types, brush up on your coding (especially libraries like Pandas and PySpark to work with data frames), etc. The joy and terror of data engineering is that it’s so wildly different for each implementation based on what you’re trying to build, but those basics don’t change much (general ideas of how to build a data warehouse haven’t changed in like 40 years).

                                      1. 1

                                        I’m on a federal project, so it’s a good long time before I’ll see any of these get approved, but definitely excited to see what’s out there on the horizon as a couple of these are very promising to me when they mature.

                                        Thanks!

                                        1. 25

                                          OTOH, the solution was presented in a direct, non confrontational way. I’m not sure I see what the problem is.

                                          Admittedly, I grew up on this sort of rhetoric.

                                          I think my issue is that the asker shouldn’t feel entitled to empathy. It is nice, but it should be extended only if the answerer wishes to. Compelling it sort of negates the whole thing.

                                          1. 23

                                            It’s not a problem per se, but when I answer a question for someone it’s normally because I care about helping them out. If I can slightly tweak the way I respond while spending little (or even no) more effort to come across as friendly then that sounds like a win.

                                            1. 8

                                              What I’m hearing is that to you, asking someone to add empathy instead of doing it naturally sounds disingenuous. I certainly understand that if you grew up this way, but most people do not. I certainly agree with you when it’s over the top and often attached to a help desk script or chatbot, but I don’t believe that’s the entire gist of this.

                                              What I got out of the article is the idea since it’s difficult to show emotion online that is conveyed nonverbally in person, extra effort goes a long way. Given that many folks asking questions may be a new to an area of study and often are at peak frustration when asking a question online. If you’re willing to help, then showing a tiny bit of extra effort to not push them entirely away from the field is a good thing. Chances are, they know a bunch of things you don’t know about and you may need help someday.

                                              So yes, I can see where you may not see a problem, but remember that large segments of people enter our various fields, or never even study long enough to bother entering, because they feel left out due to multiple gatekeeping methods. If we can make a tiny improvement here and there when answering questions to not be gatekeepers, even unconsciously, this will go a long way. Not an accusation of you personally, as I don’t think that’s your intent, but that is what I got from the post.

                                              1. 13

                                                I’m not sure I see what the problem is.

                                                That approach only really works well for the tiny subset of people like you and I who grew up on it.

                                                Compelling it sort of negates the whole thing.

                                                At work, you are compelled to perform your job function.

                                                As a senior developer, that job function typically includes improving the junior staff. You’ll never be as effective as you could in that function without cultivating a habit of empathy.

                                                It’s fine not to be high-impact in all areas, but ‘how do I teach people effectively’ is a topic that hasn’t changed in thousands of years, and it’s pretty clear that the student needs to feel good about what they’re learning.

                                                1. 9

                                                  At work, you are compelled to perform your job function.

                                                  This gives away the plot.

                                                  It’s fine not to be high-impact in all areas, but ‘how do I teach people effectively’ is a topic that hasn’t changed in thousands of years, and it’s pretty clear that the student needs to feel good about what they’re learning.

                                                  Much education has historically focused on challenging students, from Plato to Gateless Gate to military schools and boot camps. A segment of American corporate culture is becoming like a segment of American academic culture where acceptable speech, behavior, etc. are redefined in line with new norms. This ‘empathy’ push is part of that cultural change. I personally have no interest in helping the professional managerial class jockey for power in this crumbling capitalist world-order.

                                                  1. 9

                                                    Much education has historically focused on challenging students, from Plato to Gateless Gate to military schools and boot camps

                                                    An excellent way to ensure someone feels good is to set them a task they know is difficult. Not giving people access to challenging work is a great way to ensure they feel like crap and don’t learn anything.

                                                    A segment of American corporate culture is becoming like a segment of American academic culture where acceptable speech, behavior, etc. are redefined in line with new norms. This ‘empathy’ push is part of that cultural change.

                                                    I agree that cultural change does include an empathy push. I don’t think that’s the troublesome part of it (specifically, for reasons I do not understand, there’s been a huge push to ensure all students pass all subjects, with disastrous consequences).

                                                    I personally have no interest in helping the professional managerial class jockey for power in this crumbling capitalist world-order.

                                                    I’ve quit the big corporate space and am not going back. One of the biggest things I hated there was commands disguised as kind questions, so I can see your point there.

                                                    Still, I feel like the core of the argument - make an additional effort to be kind to the people you are working with, even when you are in a hurry to do something else - is worthwhile. Viewed on a scale of years, your network is far more valuable to you personally than any of the work you are doing.

                                                    1. 0

                                                      This gives away the plot.

                                                      Sorry for #me-too trashy comment but omg that was for sure the best thing I read today. I lol’d

                                                    2. 0

                                                      At work, you are compelled to perform your job function.

                                                      As a senior developer, that job function typically includes improving the junior staff. You’ll never be as effective as you could in that function without cultivating a habit of empathy.

                                                      Empathy isn’t a habit or something you need to ‘cultivate’ by pretending to be nice online.

                                                      It’s fine not to be high-impact in all areas, but ‘how do I teach people effectively’ is a topic that hasn’t changed in thousands of years, and it’s pretty clear that the student needs to feel good about what they’re learning.

                                                      How to teach people effectively has changed drastically in even the last couple of decades…

                                                      1. 11

                                                        Empathy isn’t a habit or something you need to ‘cultivate’ by pretending to be nice online.

                                                        I’m not sure what you are getting at. It is very much a trainable skill and skills are trained by making them a habit.

                                                    3. 3

                                                      There isn’t a problem, there’s a massive opportunity for growth

                                                    1. 3

                                                      Google and StackOverflow. I remember having stacks of technical books and parsing through results from Altavista + Hotbot + Yahoo! and the various product specific forums I used to have to read. It was not pleasant.

                                                      Meetups and other avenues that forced me outside of whatever industry I was working in.

                                                      20 years of experience. I am where I am now because I’ve kept learning and failing.

                                                      Doubling down on improving communication and mentoring skills. I am where I am now because I can understand what has to happen, communicate complicated issues to non-technical folks and also delegate out small stuff to let me focus on big stuff.

                                                      1. 1

                                                        Speaking about DataOps at what was supposed to be my first time at a conference, but instead will be my first webinar.

                                                        1. 7

                                                          Trying to keep my head afloat at work and make sure my family and I are good.

                                                          Checking in on my black and other POC friends and seeing what I can do.

                                                          Social distancing still, although a bit relaxed, because I’m so tired.

                                                          Checking in with family affected by the floods in Michigan last week.

                                                          Finishing up slide changes and prepping for a webinar I’m giving in a couple weeks in lieu of a conference talk that got cancelled and then went virtual.

                                                          But mostly checking in with people and trying to take care of myself and my family.

                                                          1. 0

                                                            Yes.

                                                            1. 4

                                                              I might go for a bike ride if the weather and my schedule allow it.

                                                              I’ll hopefully be doing our usual tabletop RPG night with friends on Discord (mixed blessing because we now have friends from other states involved, but also just not the same as being at a real table).

                                                              I’ll likely be working on Saturday because I’m working on stuff that requires downtime in our database.

                                                              Otherwise, I’m not touching a computer or work or anything else if I can avoid it because having a family means I’m not busier during this pandemic that I am on a normal three day weekend. Certainly not going to make a real difference otherwise. No vacation. Can’t run a lot of errands with things closed. Just… experiencing time and whatever struggle I have for the day that isn’t my blocked off time.

                                                              1. 2

                                                                This is one of those really good reads that was also wildly different than I expected it to be.

                                                                I was thinking this was going to be more data science/data engineering related (I’m a data engineer so no surprise that was my default), but as a person who programs (now data processing, previously web development), this is a good legit intro to handling application data and why they’re different.

                                                                1. 1

                                                                  Trying to keep my last shreds of sanity while maintaining social distancing, keeping up with even 80% of my work, and trying to help my wife with parenting and general household duties while also checking up on friends and family.

                                                                  1. 8

                                                                    This week I’ve been procrastinating. Sigh.

                                                                    Sometimes programming is so hard. Today I had to choose between evils, and deadlocked, typing nothing for three hours. Scrolling hither and thither. Eventually I settled for repeating some code “unnecessarily”. It works but those three hours are gone and I feel bad.

                                                                    1. 3

                                                                      These things happen to everyone, and have since the first proto-human was banging rocks together to make more useful rocks. Forgive yourself for it, think about how to avoid the deadlock next time… and then move on, change gears, and do something that makes you feel happy and productive. If you weren’t good at this stuff you wouldn’t be taking it so hard.

                                                                      1. 2

                                                                        Sometimes I just can’t get anything done.

                                                                        Sure, I come into the office, putter around, check my email every ten seconds, read the web, even do a few brainless tasks like paying the American Express bill. But getting back into the flow of writing code just doesn’t happen.

                                                                        These bouts of unproductiveness usually last for a day or two. But there have been times in my career as a developer when I went for weeks at a time without being able to get anything done. As they say, I’m not in flow. I’m not in the zone. I’m not anywhere.

                                                                        This was the guy behind VBA, Trello, and Stack Overflow talking. So don’t beat yourself up. It happens to all of us, including the great.

                                                                        1. 2

                                                                          That’s part of the job. I’ve been at this 20 years professionally and both paid and hobby work in highschool/college. I’ve had so many things that are 1-2 line fixes that too HOURS of this before I figured it out and wrote something.

                                                                          And that’s on a “normal” day, let alone when there’s an actual crisis to be managed and people who are working are often now scared because their friends are getting laid off.

                                                                          Go easy on yourself.

                                                                          1. 2

                                                                            When I feel this way (often) I try to remember a quote I like from Fred Brooks:

                                                                            The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination.

                                                                            1. 2

                                                                              That’s a great quote. Which book? I don’t recall it from the Brooks book I have.

                                                                              I think you might enjoy this image, or perhaps enjoy isn’t the right word. I didn’t enjoy it at all. Appreciate yes, enjoy no. I was doing some particularly intricate programming on the day I first saw it, and it hurt.

                                                                              I don’t know who made it.

                                                                              1. 1

                                                                                His most famous — The Mythical Man-Month

                                                                            2. 2

                                                                              I’m gonna add my voice to the crowd and say that we all have one (or more) of these days. It’s part of “a process”, I guess.

                                                                              Many years ago I arrived at the office and I found one of the lead devs of the team I was in goofing around with some music tracker. He said he’d tried to code that day, failed miserably, and it was just one of those days when it was not gonna happen. So this guy, one of the smartest and most disciplined people I’ve ever known, who – if the situation required it – would sit down at 8 AM and crank out code until 6 AM, spent the rest of the day listening to 1980s tracker music and metalcore.

                                                                              He thought (and today I think so, too!) that forcing code out of a brain that has run out of code steam will generally result in bad decisions that will take much longer to fix. Sometimes you just gotta stop and binge watch something or play or read or whatever.