Specifically to a Microsoft stack this specific pain point is outright negated, in fact, Microsoft's answer (Visual Studio SQLPROJ/Data Dude) exposes migrations for the horrible metahistory hack that they are. SQL projects are stored in source control as though they were C++/C#/what-have-you. Just like the rest of our code, the 347kloc SQL portion is handled on by CI and the migration (as well as the creation) scripts are generated for us - based on the schema differences (that it determines for us) between our last release and the current release.
So far as SQL Server goes, maintenance and versioning woes are an outright myth.
More pros:
4. A policy of using SPROCs only is a good way to significantly reduce the risk of SQL-related vulnerabilities. If developers are required to justify why they are sending raw queries to SQL they will have a hard time introducing e.g. an injection vulnerability.
5. A lot of the overhead of query execution (parsing, resolving objects, etc.) is done when you CREATE or ALTER the procedure, this can results in a performance benefit (especially if the application is chatty).
More cons:
5. Query plans for SPROCs are aggressively cached, in extremely rare circumstances this can play havoc with performance (direct contradiction of my own pro 5).
6. Triggers+family are invisible logic. They can cause confusion during debugging.
I cannot stress enough the word immediate. Delayed reports (after a week, after a day, or whenever I check) sure give a bit of reality-check and insight on what to change (usually only the first time) but were not useful for actually implementing these changes. (I have ADHD if that matters. For some people insight might be enough.)
Qbserve displays constantly the current productivity score. I had a love&meh relationship with RescueTime (since it does not offer a way to display productivity tracking all the time), but after composing it with a link to report on a new tab (my most common way of procrastination), it was a game-changer.
A small note - if you prefer something more private, and open-source, there is https://github.com/ActivityWatch/activitywatch (I use it as well). Not as polished, but clearly logging well, and it has Web UI on localhost, thus can be used with Custome New URL Tab as well.
there is the canonical Fundamentals of Wireless Communication (https://people.eecs.berkeley.edu/~dtse/book.html) which is pretty good. also, iirc, there is a book by oppenheim as well (signals and systems, i think)
Best thing is to just jump in and do it. You don't even need hardware to start with.
If you're seriously interested, download Octave (Matlab replacement) and write a program from scratch to simulate a BPSK system and generate a "waterfall" (bit error vs. noise) curve. Don't stop until you can get your curve to perfectly match the one in a text book [1]. By doing this, you will learn about modulation, demodulation and and "Additive White Gaussian Noise" (AWGN) channel models. If you can get the waterfall curve to match exactly, you will also be covering concepts such as "energy per bit". A BPSK should be quite doable for an amateur.
Once you have the BPSK simulation, try extending your simulation in different directions and matching the textbook curves. The following challeges are roughly in order of increasing difficulty
1) Add a block code, such as Hamming or BCH.
2) Replace the simple AWGN channel model with a Rayleigh or Ricean model
3) Add a convolutional code (with Viterbi decoder)
4) Implement Orthogonal Frequency Division Multiplexing (OFDM)
5) Implement an LMS receiver
6) Implement a Low-Density Parity Check code (use the one from the WiFi standard)
7) Implement a complete 802.11a simulation
8) Implement a complete DVB-T simulation
9) Implement a MIMO system, assuming perfect channel knowledge
10) Implement a MIMO system with channel estimation
Get to number 10, and you will know more about radio signal processing than most engineers.
[1] eg. "Lin and Costello" Error Control Coding or "Proakis" DSP
> This Economist article points out some of the many small academic works that quibble over details with Piketty and Saez. But that's not anything new. The major points of their work, and especially of Piketty's monumental _Capital for the 21st century_ still stand: that capital is a positive feedback loop in a way that labor is not; that mid-20th-century laws that put brakes on this feedback loop have been removed; that a variety of data sources are confirming growing inequality and market capture particularly in the UK and US; and that the only reasonable solution for this is a tax on owned capital (not on income or cap gains) and the political chances of this happening are slim, etc. [emphasis added]
I'm a bit surprised that you included ad hominem accusations of bias and haterism given the adjectives in the first paragraph. :)
But, in seriousness, Piketty's work is pretty poor. The empirical data is now discounted [0], the `r > g` claim was debunked shortly after publication [1]. The fate of the latter claim seems fatal to Piketty's attempts to accurately model the world and prescribe solutions. For instance, capital depreciates as it ages, and can do so dramatically due to obsolescence or damage. Redeploying capital toward new uses is generally quite costly, due to the high inelasticities of capital. Labor is much more elastic to different tasks of production. (I should note that the inelasticity of "human capital" in the modern service-based economy, which is indeed a problem. "Learn to code" is a meme deployed by trolls, but it reflects a real economic reality. The increasing prevalence of human capital and its inelasticities also undercuts Picketty.)
Further, Piketty himself defines capital in such a way (he uses it essentially synonymously with wealth) that his argument is almost tautological, and also fails to account for wealth stored in e.g. real estate (or the works of Picasso, for that matter). Increases in real estate prices (driven almost entirely by well-understood microeconomic structural issues and not by some abstract macro mathematical inequality) account for the amount r exceeds g.
Any usual definition of capital emphasizes that it is a factor of production, not merely wealth. Capital is the fixed "stuff" we use to produce other stuff. It might be a dump truck or an assembly line process or even knowledge of C++. Housing stock almost certainly fails the factor of production test, and thus shouldn't be counted as capital (the status of a Picasso as a factor of production is an exercise left to the reader). (By analogy, labor is also defined as a factor of production, and doesn't include the value of leisure hours or time spent sleeping.) Piketty's book sales might not have been so high had he simply argued that fixed supplies of highly-demanded scarce resources tend to increase over time, and that land in a reurbanizing and NIMBY-ish period is by far the strongest such asset. Henry George said more insightful things about the same subject over a century before Piketty had an economics professorship. (I'm not arguing that George's solutions were great. I'm more enthusiastic about ideas like nuisance-based zoning as opposed to use-based: they seem to achieve the desired effects of stability and low inequality without a ton of extraction. I somehow doubt that Piketty would share this enthusiasm. [2][3])
Ultimately, if these small attacks of death by 1000 cuts on Piketty's work don't convince you (and I would characterize some of the attacks as quite substantial) that it was very flawed and probably just nonsense, then maybe ask yourself if any evidence whatsoever would convince you that Piketty is (broadly speaking) wrong.
Two decades ago I was overjoyed to discover that Scheme was finally going to have a useful application beyond illustrating SICP and writing koans to amuse myself, because DSSSL was on the cusp of evolving into the last document styling language anyone would ever need.
Unfortunately following an incident with a broken Lisp machine, a liquid lunch, and an unlicensed particle accelerator, I became trapped in a parallel universe where the HTML ERB anointed CSS by mistake during a drunken night out in Oslo.
The fundamental concept of CSS (best revealed by H.W.Lie's thesis IMO[1]) was to create a rich and versatile and non-Turing-complete set of structural selectors in lieu of DSSSL's recursive logic, and to allow styles to overlay one another; two design choices that only by the application of gallons of irony can explain why most web pages are composed of a bunch of nested DIV elements with hashed IDs and overloaded semantic class attributes, and everyone compiles their assets into a static file.
I switched to Tailwind CSS months ago. Adam Wathan is the hero we deserve.
On a recent project, it was a weird inversion in terms of access... in order to keep the middle tier thin, and meet requirements that all data access happen through stored procedures... we pretty much standardized an interface with one input parameter (@json) and two output parameters (@result, @errorResult). In the end, all input/output was JSON and the database handled all data internally.
I don't really like it much, but it did make the API service super thin, and the "database guys" were happy to do it all in SQL. Of course, testing that beast isn't so fun, and there be dragons...
In the end, I tend to favor creating APIs with scripted languages that require less translation to work with the database side, and structure responses to match the expected data layouts for the API side. With node, I usually create/use a simple abstraction...
var result = await db.query`SELECT ... WHERE foo=${bar}`;
or
var result = await db.exec('sprocname', {...});
In either case, not really a need for a formal ORM here.
For simple "web apps", typically its a poor understanding of SQL that is the bottle neck. Since most web frameworks include an ORM, many developers don't even know much SQL any more and treat the database as a simple 'object' that you can call methods on and iterate through. Case in point: you're building a dash board for a web based store to show details about the 5 most recent orders. But each order references 5 other items (as a simple example). And each item references an SKU# that can come from 5 manufacturers.
So an optimized query set from an ORM will end up querying the DB like this: First it selects the 5 rows from the order table, but since you left out any join clauses, it then runs 5 more selects on each of the rows returned from the order table to pull the related rows from the items table. So we are now up to 6 queries.
But each row of the item table references 5 manufacturers (again, as a simple schema example). So each of the 'item' 'objects' (5 per order) has to then make a query to fetch the sku manufacturer. So that is 25 more select statement queries on top of the original 6. So using the ORM properly and building out the right join clauses, you would have made 1 query. Instead you made 31. Now lets say that DB is not on the same machine, but a different machine on the network. Throw in 2ms - 10ms latency per query and now you are looking at 62 - 310ms of latency to fetch your data. And that is just for 5 'rows' of data. Do the math for a more realistic dataset, 500-1000 rows or so. Now you're looking at minutes of latency for your data to come from the DB.
The worse case I've seen recently was this exact case. To generate an inventory page to show an html table with about 300 rows, the DB was being queried over 300,000 times with a page load of about 7 minutes (and getting longer everyday). I added the proper join clauses to the ORM and one query was issued which would return the result set in about 700ms which put the page load time to about 1 second. A 1 second page load imo is still not very good. But it seemed to impress everyone that with 1 hour of work, a page load went from from 7 minutes to 1 second....
For what ever reason, most people don't understand this anymore. Instead of learning SQL, the line of thought is to upgrade the machine, move to a 'no-sql' DB, introduce some caching layer, and many number of other ideas. But "lets make sure we are querying the DB instead of iterating through it" is never discussed.
That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places. If they'd had the data, they could have spun up their service on a different hosting provider relatively easily.
You don't need to use containers for that.. all you have to do is set up a warm replica of the service with another provider. The fail over doesn't even have to be automatic, but that is the minimum amount of redundancy any production SaaS should have.
I think the technical security problem here (properly whitelisting parameters in logs) is just a symptom and not the core underlying concern (as the article mentions Twitter and Github just dealt with similar issues)
To me, it seems very likely someone before 2019 laid eyes on these logs and either:
a) Decided not to report it (implies serious security culture issue)
b) Reported it and no action was taken (implies serious security process issue)
c) Didn't even acknowledge it was inappropriate (implies a serious security training failure)
If you've already become complicit towards regularly violating the privacy of your end-users, one can easily understand an employee devaluing the seriousness of clear-text passwords in a log.
Are FB employees so regularly exposed to sensitive data that they have become desensitized to the seriousness of clear-text passwords in an internally accessible log?
I mean sure, if you have limitless time then feel free to learn a bunch of programming languages as well as everything else you could want to learn. But real humans have opportunity cost. If you are a software engineer, the time you spend on skill development can already cut across many, many dimensions. Additional programming languages is just one, and I'd argue a fairly narrow one after you've touched on a few key language paradigms.
What about:
- Architecture
- Software Delivery
- Networking
- Project management
- Interaction design
- Visual design
- Human factors/Social systems
- Graphics/Art (2D/3D)
- Market validation
- Sales & Marketing
- Business planning and finance
- Application domain knowledge
- Operations and Monitoring
- Data infrastructure
- Analytics
- Machine Learning
- Machine Vision
- Computer Graphics
- Simulation
- Game Engineering + Design
- Information Retrieval + Recommender Systems
- Embedded systems/Control theory
- Optimization
- Scientific Computing
I mean there is basically an endless list of areas you can reach into in the limited time you have time focused on skill development as a software engineer beyond "on the job" training. Building a strong, broad "stack" of skills seems like a good investment. Learning new programming languages is a niche within a niche depending on how expansive a scope you set for yourself as a person creating software for the world to use.
> This code has to be maintained, versioned, etc.
Specifically to a Microsoft stack this specific pain point is outright negated, in fact, Microsoft's answer (Visual Studio SQLPROJ/Data Dude) exposes migrations for the horrible metahistory hack that they are. SQL projects are stored in source control as though they were C++/C#/what-have-you. Just like the rest of our code, the 347kloc SQL portion is handled on by CI and the migration (as well as the creation) scripts are generated for us - based on the schema differences (that it determines for us) between our last release and the current release.
So far as SQL Server goes, maintenance and versioning woes are an outright myth.
More pros:
4. A policy of using SPROCs only is a good way to significantly reduce the risk of SQL-related vulnerabilities. If developers are required to justify why they are sending raw queries to SQL they will have a hard time introducing e.g. an injection vulnerability.
5. A lot of the overhead of query execution (parsing, resolving objects, etc.) is done when you CREATE or ALTER the procedure, this can results in a performance benefit (especially if the application is chatty).
More cons:
5. Query plans for SPROCs are aggressively cached, in extremely rare circumstances this can play havoc with performance (direct contradiction of my own pro 5).
6. Triggers+family are invisible logic. They can cause confusion during debugging.