Stuff The Internet Says On Scalability For September 21st, 2018 - High Scalability -

 highscalability.com  09/21/2018 16:06:40 
  • Quotable Quotes:
    • Linus Torvalds: This week people in our community confronted me about my lifetime of not understanding emotions.  My flippant attacks in emails have been both unprofessional and uncalled for.  Especially at times when I made it personal.  In my quest for a better patch, this made sense to me. I know now this was not OK and I am truly sorry.
    • @jessfraz: To those in my mentions. This "foundation" which has tax breaks... Is paying people who don't contribute to the kernel 400k a year. Would you like to rethink your "they build the kernel" comments.
    • Aria Bracci: Using a previously established, peer-reviewed technique, the team conducted more than half a million data traffic tests across 161 countries. From this data, the team found that internet service providers are “giving a fixed amount of bandwidth—typically something in the range of one and a half megabits per second to four megabits per second—to video traffic, but they don’t impose these limits on other network traffic.”
    • @mjpt777: "Patterson indicated that rewriting Python into C gets you a 50 times speedup in performance".
    • @JoeEmison: The idea that any kind of planning will necessarily lead to overengineering is part of the anti-planning developer culture that misunderstands the agile manifesto.
    • Marie Hicks: The dynamic continues to this day. Silicon Valley reaps enormous profits at the expense of the majority of users, and calls it progress. But technology’s alignment with actual progress has a long and uneven history, and its effects are rarely straightforward or fully foreseen. Real progress isn’t synonymous with building another app—it involves recognizing the problems in our society and confronting the uncomfortable fact that technology is a tool for wielding power over people. Too often, those who already hold power, those who are least able to recognize the flaws in our current systems, are the ones who decide our technological future.
    • @0xmchow: I compiled a very short list to illustrate how long we've been playing this "cyber security skills/workforce shortage" game for [link] …. This game has been going on for a long time and is becoming "boy who cried wolf."
    • Strachey: It has long been my personal view that the separation of practical and theoretical work is artificial and injurious. Much of the practical work done in computing, both in software and in hardware design, is unsound and clumsy because the people who do it have not any clear understanding of the fundamental design principles of their work. Most of the abstract mathematical and theoretical work is sterile because it has no point of contact with real computing.
    • @EmilyGorcenski: Excuse me, I have been a government software contractor and I assure you this is not the largest man-made waterfall
    • Nikita: You don’t have to be a genius to write fast programs. There’s no magic trick. The only thing required is not building on top of a huge pile of crap that modern toolchain is.
    • David Patterson: We are now a factor of 15 behind where we should be if Moore’s Law were still operative. We are in the post–Moore’s Law era...single program performance only grew 3 percent, so it’s doubling every 20 years. If you are just sitting there waiting for chips to get faster, you are going to have to wait a long time...Revolutionary new hardware architectures and new software languages, tailored to dealing with specific kinds of computing problems, are just waiting to be developed. There are Turing Awards waiting to be picked up if people would just work on these things.
    • Vint Cerf: differential traceability. The ability to trace bad actors to bring them to justice seems to me an important goal in a civilized society. The tension with privacy protection leads to the idea that only under appropriate conditions can privacy be violated. By way of example, consider license plates on cars. They are usually arbitrary identifiers and special authority is needed to match them with the car owners ... This is an example of differential traceability; the police department has the authority to demand ownership information from the Department of Motor Vehicles that issues the license plates. Ordinary citizens do not have this authority.
    • Dr Denis Bauer: We started out with one infrastructure [for genetic analysis] and then tweaked it to get better performance and do the analysis we wanted to do. I strongly believe that once you go serverless you never go back.
    • Herb Sutter: Perhaps the most important thing we can do for C++ at this point in its evolution is to make sure we preserve its core strengths while also directing its evolution in ways that make it simpler to use. That is my own opinion at least, and so this talk starts with a perspective question: What “is C++,” really? The language continues to evolve and change; as it does so, how can we be sure we’re picking C++ evolutionary improvements that not only don’t lose its “C++-ic” qualities, but make it a better C++ than ever?
    • S. Brent Rodriguez-Plate: As philosophers like Andy Clark have argued, “our tools are not just external props and aids, but they are deep and integral parts of the problem-solving systems we now identify as human intelligence.”
    • Mark Ames: The secret wage-theft agreements between Apple, Google, Intel, Adobe, Intuit, and Pixar (now owned by Disney) are described in court papers obtained by PandoDaily as "an overarching conspiracy" in violation of the Sherman Antitrust Act and the Clayton Antitrust Act, and at times it reads like something lifted straight out of the robber baron era that produced those laws. 
    • nemothekid: In my experience, yes network partitions are incredibly rare. However 99% of my distributed ststem partitions have little do with the network. When running databases on a cloud environment network partitions can occur for a variety of reasons that don’t actually include the network link between databases: 1. The host database is written in a GC’d language and experiences a pathological GC pause. 2. The Virtual machine is migrated and experiences a pathological pause 3. Google migrates your machine with local SSDs, fucks up that process and you lose all your data on that machine (you do have backups right?) 4. AWS retires your instance and you need to reboot your VM.
    • @GossiTheDog: I should point out with an organisation that size [Equifax], which is effectively a series of large different businesses linked by a patent (and WAN) it is difficult to know what is going on. But if you have passwords in plaintext in CSV files things are probably not to policy.
    • @mipsytipsy: Anyway.  What you need to build an elite engineering team is not lots of elite engineers, but a team that is willing to iterate on their tools and processes. Care about user experience.  Double down on what works.  And choose good tools to help you get there.
    • @QuinnyPig: No, AWS. They’re @harrys. They didn’t “reduce” their operations cost, they shaved it.
    • Ezra Callahan: The single greatest growth mechanism ever was photo tagging. It shaped all of the rest of the product decisions that got made. It was the first time that there was a real fundamental change to how people used Facebook, the pivotal moment when the mind-set of Facebook changes and the idea for News Feed starts to germinate and there is now a reason to see how this expands beyond college.
    • @dougtoppin: That moment when it is Fri evening, you push your serverless application model to AWS CodeCommit, CodePipeline triggers CodeBuild, your new stuff deploys to Lambda and API Gateway, your Postman tests successfully run and you run out before something goes Kablooey! AWS SAM is cool
    • Wired: But a big California farmers’ lobbying group just blithely signed away farmers’ right to access or modify the source code of any farm equipment software. As an organization representing 2.5 million California agriculture jobs, the California Farm Bureau gave up the right to purchase repair parts without going through a dealer. Farmers can’t change engine settings, can’t retrofit old equipment with new features, and can’t modify their tractors to meet new environmental standards on their own. Worse, the lobbyists are calling it a victory.
    • Charles Schwab: The trade deficit is being used as a yardstick, as a rationale for moving into this trade war or trade skirmish, as we’ve been calling it. Yet at the same time, tax cuts have helped to boost the economy, boost consumer spending, which, in turn, will widen out the trade deficit, because we simply buy more imported goods. Yet a widening trade deficit may be used as a rationale for more tariffs. 
    • @mipsytips: The most effective way to structure your instrumentation, so you get the maximum bang for your buck, is to emit a single arbitrarily wide event per request per service hop. We're talking wiiiide.  We usually see 200-500 dimensions in a mature app.  But just one write.
    • @kchoudhu: Super important distsys thread. I implemented fat logging my last job for a pipeline handling millions of financial transactions a day. L3 support requests went down 82% after people lower down in the support hierarchy gained god's eye visibility of single events.Kamil Choudhury added,
    • @codinghorror: I need to find time to blog this, but TL;DR we've gotten *insane* value for the dollar out of our public vulnerability bug bounty program, so much so that it has completely changed my mind on the matter
    • @brianleroux: Avoiding "vendor lock in" with AWS by building your own functions as a service on a container platform with a cloud provider that has *at best* 1/4th the revenue/customers/scale. Bonus: you also get to avoid 100% utilization!
    • SoulMan: My company is was full on into GCP and after a year of terrible support management suddenly decides to go all in into Azure. We use every managed service from GCP including BigQuery which is nearly impossible to find a replacement of in Azure. Appengine needs to be converted to Azure Kubernetis Service. So its not a technical decision completely, just that company signed a bigger deal with Microsoft and wants to pay a single bill. Based on feedback from other teams company feels Azure support is way more professional when it comes to production outage. Now all of 3 years work sing Google services needs to be re-written in "cloud agnostic" way.
    • sgt101: Lock-in occurs when you no longer can generate the capex to resolve the opex drain that a vendor squeezing you has created. If any vendor realises that you can raise the capex to get off them then they will raise the opex demand to the point of financial viability to maximise their returns and to ensure future returns (because by doing this they underline their control)
    • Spotify: Read requests to Cassandra are cached on a memcached cluster. (For reference, you can find Folsom, an open source version of our client here.) The p99 request latency to our memcached cluster is between 2 and 4 milliseconds, while the request latency to our Cassandra cluster is around 10 milliseconds. When serving traffic without cache, Cassandra request latency grows to a p99 of about 15 milliseconds. Thus, having a memcached cluster for caching lowers our p99 by about 10 milliseconds.
    • eslaught: One of the central insights of Legion was to [explicitly reify the notion of data partitioning](http://legion.stanford.edu/pdfs/oopsla2013.pdf) in the programming model. Sequoia didn't need this because it simply knew the inputs and outputs to every task. Legion couldn't afford to pay the $O(N \operatorname{log} N)$ cost to perform this analysis at runtime. By lifting partitioning into a first-class primitive, we were able to radically reduce the runtime cost of this analysis by leveraging what the user already know about their data partitioning. Originally this required users to declare certain properties of their partitions, such as disjointness, in order to make things efficient. Eventually we discovered a [partitioning sublanguage](http://legion.stanford.edu/pdfs/dpl2016.pdf) which allowed us to statically discover most of these important properties, and to provide many static guarantees even in the presence of data-dependent behavior.
  • « Go back