1. 3
    1. 2

      Thank you

      1. 2

        Oh thank you

      1. 1

        Just curious guys How’s your experience has been with blackblaze I’m leaning towards that

        1. 1

          I just keep buying physical storage.

          1. 1

            Damnn i see I just. Can’t buy physical storage at the moment

          1. 2

            Backblaze B2 is an excellent, reliable, widely-supported, and relatively cheap option for blob backup storage. Works with tools like arq (OSX, Windows), restic (Linux), rclone, etc.

            Usually your best bet is to store your personal data in some sort of NAS or attached storage and then mirror to something like B2 off-site, perhaps with encryption for extra peace of mind.

            I personally find that using rclone with Backblaze B2 is the best for performance and ease.

            https://rclone.org/b2/

            With rclone, you can easily saturate your network uplink and you can do partial/incremental uploads with minimal fuss. You can also configure it to not use you full uplink straightforwardly, to spare the bandwidth for your other home uses.

            https://rclone.org/docs/#bwlimit-bandwidth-spec

            1. 1

              I’ll check out blackblaze B2 Btw what’s the difference between there personal storage and b2 I’m just can’t buy nas at the moment

            1. 6

              I currently use NextCloud (self hosted), with offsite backups using tarsnap.

              1. 4

                The downside of tarsnap is that once you lose your key, you lose your backups. For some cases such strong crypto is warranted, but for family pictures? I’m not so sure, at least not for me personally.

                I say that as someone who lost access to his backups after I lost access to my backups shortly after both my machines where I had the passphrase stored were lost. There’s been a bunch of stories like this with strong encryption, people have three copies of their keys/passphrases and still manage to lose access.

                Just something to keep in mind.

                1. 1

                  Yeah, that is something to keep in mind. I’ve backed up my key (converted to QR, printed, laminated, and stored in a separate secure location).

                2. 1

                  I see I’m just scared to host it my self just scared about the security since I’ll be hosting family pictures

                1. 1

                  I’m saving this thread for the future since I’ll be switching fro Linux to macbook. Can anyone suggest me how to get a tiling window manager on macbook

                  1. 3

                    You can install Linux or BSD on a Mac, or you can virtualize to run them, but for macOS itself there is no ecosystem of alternative window managers and no such abstraction layer. macOS does not run X.org or have any similar abstraction layer for multiple window managers; it has its own window server that’s tightly integrated with the Mac SDK.

                    1. 2

                      I haven’t been able to find a tiling window manager on the Mac that doesn’t end up feeling like fighting the platform. I use them exclusively on X, but I don’t bother any longer on the Mac.

                      1. 1

                        I’ve not used them myself, but Amethyst and yabai are options.

                      1. 3

                        I use Hyde (http://wiki.call-cc.org/eggref/5/hyde) for my personal website (https://avalos.me/) and Hugo for my blog (https://blog.avalos.me/), using the Etch theme, which is pretty minimal and nice. I’m starting to think on moving my blog to Hyde as well, so I can have it along my personal website.

                        1. 2

                          That’s pretty cool. I am using a similar theme for hosting my notes and things I learn across the internet https://knowledge-book-six.now.sh/

                        1. 1

                          I must say I am loving reading everyone’s replies and it gives me a different perspective of how something you love can help you land a job such as helping in software communities , writing a blog or doing side projects. Leetcoding and side projects both have pros and cons but I would prefer side projects as it really tests your abilities

                          1. 3

                            I have pretty good github page: https://github.com/antonmedv It helps significantly.

                            1. 1

                              You have a lot of stars. May I ask how do you promote your projects

                              1. 2

                                reddit/lobster, rarely hacker news. Sometimes one/or another got popular. I think the main inside - I do lots of projects (it’s a hobby for me). Example:

                                And, of cause, I have lots of unpopular projects:

                                As you can see, quantity grows into quality.

                            1. 4

                              All sorts of things. A side project turned into an (unsuccessful but educational) company, at the same time that I changed careers from geology to programming. I had nothing else recent on my resume that was programming-related.

                              Another side project got somewhat popular and so when I posted about quitting my current job a collaborator said “hey, why don’t you work for me as a contractor for a while?”, and several other interested people reached out to me with interviews. The contractor gig not terribly successful, but again very educational, and so I went on as a contractor by myself for a while, because I knew I could do it. That side project also resulted in a lot of documentation and testing work, so when I moved on to the next job the reception was “wait, you know how to do all this stuff? PLEASE KEEP DOING IT”. I also know a lot more about robotics than I really expected, because I like writing video games on the side, and turns out a lot of the math and problem-solving is very similar.

                              Do something you’re interested in, treat it professionally, and some part of it WILL be useful someday down the line.

                              1. 1

                                Agreed. I totally believe in this and this is also a way of giving back to the community

                              1. 1

                                I’d say I first heard about the posting for my current job from a tech forum I answer questions on, and may have got the interview because I would help people with their problems.

                                The job is embedded development, something I’ve been meaning to get back into. I had still been doing embedded development on my own just because it’s an interest of mine. I don’t think anyone saw my blog, but just being able to talk about embedded stuff during the interview really helped.

                                But similar to what @BenjaminRi mentions, my blog got the attention of someone working on a project similar to mine. They were much more experienced and they reached out to me and helped me through a lot. So not related to getting a job but my blog really did help me get connected with them.

                                I have no idea what leetcoding is.

                                1. 1

                                  I see many people posting having no idea what leetcoding is on here This gives me hope if you are willing to put in enough work by writing blog or even side projects this can help you get noticed and you can end up having a rewarding career. Honestly this gives me a lot of hope

                                  1. 3

                                    Success is what you define it to be. Internalize this.

                                    The archetypical tech success story involves high salaries, famous companies, and a life defined by upward mobility. This is extraordinarily fragile if only because each one of these metrics is other people’s perception of you, which you cannot control.

                                1. 3

                                  I’ve never gotten a job specifically via side project, but they have acted as good nerd-credo in interviews and with co-workers.

                                  I think this has been mentioned before, but I think the group of people who grind sites like leetcode and the group of people who have a significant side-project output is pretty separated.

                                  Side projects, appropriately deployed, have to interact with the real world in a much wider context than leetcode exercises, they’re a lot more likely to be ambiguous and deal with fuzzy scenarios and force you to problem solve in a product-oriented way. leetcode, on the other hand, has one purpose: Get better at algorithms so you can do better in interviews.

                                  Of the two, I’d highly recommend writing a side project that scratches a personal itch of yours, and then figure out how to try to share it with other people that might want it, going as far as trying to learn how to market and sell it.

                                  1. 2

                                    Agree. People are often very good at sniffing out intrinsic/extrinsic motivation, and, ironically, intrinsic motivation is often a stronger hiring indicator than extrinsic. Or, at least for the types of jobs I’m interested in. :)

                                    1. 1

                                      I’m unsure that that’s necessarily a healthy thing for the industry as a whole, (in part because for many people, myself included, intrinsic motivation can be a fickle thing). But, at the same time, I do prefer to work with people who have a curiosity for software development that isn’t bound to only the current job.

                                      Though, more than anything, I want to work with people who have made the space (at work or otherwise) to keep learning, and that don’t get stuck in a rut from 5-10 years ago. And side projects are not the only way to have that.

                                      1. 1

                                        Agree. I see it more as doing something, rather than sitting around and hoping things change for the better.

                                        It is entirely rational for employers to want experience working in certain domains, like compilers. And it is entirely reasonable for employees to want to switch into those domains even without experience. The only way around that is either academic or self-instruction. It’s not economically or personally sustainable all the time, despite what all the productivity blogs claim.

                                        But it’s a helluva lot better than feeling stuck.

                                    2. 1

                                      I totally agree with you. Leetcoding tests your algorithmic skills but It can also be memorized but Personal Side Projects really tests your creativity and helps you learn a lot. Side projects basically allows you to your idea to life.

                                    1. 3

                                      I recently purchased an @.dev domain for email and started referring to my blog on my CV (I mainly talk about math). I don’t know if it’s coincidence, but I landed a high-profile job shortly after that. I suspect it might have helped, not because my blog is especially high quality, but just because it shows that you like to spend time investigating.

                                      1. 4

                                        that’s really cool and I would love to read your blog

                                        1. 2

                                          Ooh, please link your blog!

                                          1. 1

                                            It’s rubenvannieuwpoort.nl (the site is hosted on rubenvannieuwpoort.github.io) :) The “I mainly talk about math” remark was directed at my blog, not at my CV. BTW I am currently rewriting the article about Cardano’s formula (I think it’s actually called “Cardano’s method”) and I have a lot of ideas for articles that I haven’t found the time for to write.

                                        1. 4

                                          What’s FPGA and how it’s used (I’m a beginner)

                                          1. 4

                                            It’s a “field-programmable gate array”. It uses very flexible hardware (arrays of specially interconnected look-up tables) to emulate specific hardware designs, typically written in Verilog or VHDL. These are mostly used to prototype hardware designs and verify logical correctness before physical layout and (very expensive!) manufacturing of ASICs. Occasionally these prototypes (“soft cores”) are deployed as-is in special purpose equipment, despite being slower and less efficient than an ASIC – they are much less expensive to manufacture and can be upgraded without replacing the physical hardware. They also are sometimes used to directly implement special-purpose (especially data-parallel) algorithms in logic gates.

                                            The small ones are pretty cheap now. You can rent bigger ones from AWS.

                                            1. 1

                                              Oh i see thanks for explanation I heard they are used in HFT just looked up on google

                                          1. 3

                                            Learning Powershell

                                            1. 2

                                              I recommend you check “Designing Data-Intensive Applications” from Martin Kleppmann. It’s a good introduction to many of the aspects you’ll have to deal with as a Data Engineer. Another source of good information are podcasts, I can vouch for The Data Engineering Podcast and the data section of Software Engineering Daily. Both have some really good interview with people from the industry that can shed light on how data engineering fits in different companies.

                                              1. 2

                                                Sure I’ll check them out Thank you for sharing these resources

                                                1. 2

                                                  Thank you for sharing these

                                                1. 5

                                                  Are you trying to switch technical specialties or coming in new? I’m assuming the former from your ask, but going to generalize for all readers.

                                                  If it’s the latter, I’d say most data engineers seem to come from either a standard development background focused mostly on back end work. You know how to do ORMs but are also decent at SQL and database structure, but most importantly, have been doing coding for a while.

                                                  OR

                                                  You come from an analytical background, data analysis, data science, visualization, etc. You may not be as proficient at production coding practices, but you really grok data and all its messiness, when to do an aggregated, denormalized structure vs a normalized transactional one, etc..

                                                  Either way, you need to enhance whatever skills your lacking on either the standard code side (likely in Scala or Python) or you need to get a little more familiar with various data storage types (relational databases, graph, document store, etc.) and the weirdness inherent in data that analysts and statisticians are happy to tell you about.

                                                  If you’re coming in totally cold out of a boot camp or college, then the basics are really try to understand at least the basics of SQL and how databases work (this is the basis of all the other more complicated stuff anyway), a programming language (Python is your faster bet here if you’ve never programmed), and a sampling of a few other things – data modeling, how APIs work, ORMs, what data scientists/analysts/visualization people do, how back end datasources work for standard software, a bit of DevOps, etc. The big parts are the basics of data and programming to access and process it then store it elsewhere though because that’s the foundation.

                                                  I’d go into more detail, but really, there’s so many database types, orchestration tools, frameworks, cloud services, etc. that it would make the Javascript folks heads spin to know they all exist. The rest of it is really more about the type of work you’ll end up doing, and being flexible is the best way to go about it.

                                                  Happy to answer more questions since I sort of fell into it over the years, but work with a variety of other data engineers and also make a point of teaching and spreading the word when I can.

                                                  1. 2

                                                    Thank you for writing such a thoughtful reply and It means a lot. I have mostly programmed in python and I do have the knowledge of SQL but not advance just beginner since I mostly did ORM instead of writing plain SQL queries but I would love to know more like what resource can I consume since to broaden my knowledge

                                                    1. 1

                                                      No worries. The main reason for learning SQL is it’s the basis of other query languages for non-relational databases. I’d say knowing the basics will get you a ways from that standpoint, and the advanced stuff you end up picking up as you do it.

                                                      I stumbled a lot into how some of this works by experience, but I’d say doing basic free tutorials for various database types (Postgres for relational, MongoDB for document, Neo4j for graph) are a good start. There’s a ton of links out there on Dev.to and various Medium sites (Toward Data Science often has some good stuff) to get you on more of the basics of data engineering.

                                                      This is also an interesting way to look at things since you’re already doing Python.

                                                      https://medium.com/@maximebeauchemin/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a

                                                      1. 1

                                                        Beyond that, I’d say look into the idea of DataOps (using DevOps, Agile, and Lean principles to work data projects) for some good way to get up to speed on the up and coming way to go. The DataOps Podcast is a great way to do that, along with resources that are put out by a few companies pushing the idea forward that have put out a ton of free info.

                                                        Also, if you haven’t found it already, the Data Engineering podcast is a good place to get an idea of general toolsets and challenges being used.

                                                        Probably a lot more to cover there – I’d say just getting the basics and a good hold on how to work with data in general as well as make sure your understanding of different data storage types, brush up on your coding (especially libraries like Pandas and PySpark to work with data frames), etc. The joy and terror of data engineering is that it’s so wildly different for each implementation based on what you’re trying to build, but those basics don’t change much (general ideas of how to build a data warehouse haven’t changed in like 40 years).

                                                    1. 1

                                                      What email platforms can you use to host discussions so everyone can see