From 1881d9018f75fd3e75bcf68e8d244a3dee7fccbe Mon Sep 17 00:00:00 2001 From: David Gasquez Date: Wed, 27 May 2026 21:22:28 +0200 Subject: [PATCH] =?UTF-8?q?=F0=9F=94=97=20Link=20related=20handbook=20note?= =?UTF-8?q?s?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Agentic Engineering.md | 14 ++++++------ Analytics Engineering.md | 8 +++---- Antifragility.md | 2 +- Asynchronous Communications.md | 6 ++--- Automation.md | 4 ++-- Blockchain.md | 6 ++--- Blogging.md | 10 ++++---- Cats.md | 2 +- Checklist.md | 4 ++-- Communication.md | 8 +++---- Company Handbooks.md | 2 +- Company Knowledge Management.md | 6 ++--- Cooperatives.md | 2 +- Coordination.md | 8 +++---- Cryptocurrencies.md | 2 +- Culture.md | 10 ++++---- Curiosity.md | 4 ++-- Dashboards.md | 8 +++---- Data Culture.md | 14 ++++++------ Data Engineering.md | 8 +++---- Data IDE.md | 10 ++++---- Data Package Manager.md | 28 +++++++++++------------ Data Practices.md | 12 +++++----- Data Quality.md | 8 +++---- Databases.md | 2 +- Datathons.md | 2 +- Decentralized Autonomous Organizations.md | 2 +- Decentralized Protocols.md | 12 +++++----- Decentralized Web.md | 6 ++--- Deep Funding.md | 12 +++++----- Design Docs.md | 8 +++---- Documentation.md | 14 ++++++------ Double Crux.md | 2 +- Dungeons and Dragons.md | 2 +- Emergence.md | 4 ++-- Engineering Process.md | 2 +- Evolution.md | 2 +- Experimentation.md | 8 +++---- Federated Networks.md | 4 ++-- Feedback Loops.md | 8 +++---- Feedback.md | 14 ++++++------ Finances.md | 2 +- Fitness.md | 2 +- Focus.md | 2 +- Future.md | 10 ++++---- Game Design.md | 4 ++-- Gardening.md | 2 +- Git.md | 2 +- Goals.md | 2 +- Governance.md | 2 +- Habits.md | 6 ++--- Hobbies.md | 4 ++-- House.md | 4 ++-- How To Do Hard Things.md | 2 +- IPFS.md | 8 +++---- Idempotence.md | 2 +- Impact Evaluators.md | 18 +++++++-------- Incentives.md | 6 ++--- Journaling.md | 4 ++-- Learning.md | 6 ++--- Life Areas.md | 4 ++-- Listening.md | 4 ++-- Machine Learning.md | 14 ++++++------ Making Decisions.md | 2 +- Mechanism Design.md | 16 ++++++------- Meditation.md | 2 +- Meetings.md | 6 ++--- Mental Health.md | 6 ++--- Mental Models.md | 10 ++++---- Metrics.md | 8 +++---- Mindfulness.md | 4 ++-- Modularity.md | 2 +- NFTs.md | 2 +- Network Effects.md | 2 +- News.md | 4 ++-- Nonviolent Communication.md | 4 ++-- Nutrition.md | 2 +- Open Data.md | 16 ++++++------- Open Source Data Projects.md | 6 ++--- Openness.md | 8 +++---- Organizations.md | 6 ++--- Pairwise Comparisons.md | 4 ++-- Parenting.md | 8 +++---- Personal Handbooks.md | 4 ++-- Piano.md | 4 ++-- Pixel Art.md | 2 ++ Planning.md | 2 +- Plurality.md | 4 ++-- Podcasts.md | 4 ++-- Privacy.md | 6 ++--- Problem Solving.md | 6 ++--- Productivity.md | 4 ++-- Programming.md | 2 +- Prompt Engineering.md | 6 ++--- Public Goods Funding.md | 10 ++++---- Public Speaking.md | 6 ++--- Quotes.md | 6 ++--- README.md | 4 ++-- Recipes.md | 2 ++ Relationships.md | 8 +++---- Remote Jobs.md | 2 +- Resolving Disagreement.md | 4 ++-- Reverse ETL.md | 6 ++--- Routine.md | 10 ++++---- Science Fiction Reads.md | 2 +- Science.md | 6 ++--- Signaling.md | 2 +- Slack.md | 2 +- Sleep.md | 2 +- Social Games.md | 10 ++++---- Spaced Repetition.md | 2 +- Systems.md | 6 ++--- Technical Specifications.md | 4 ++-- Unified Schema Design.md | 4 ++-- Web Based Tools.md | 2 +- Writing Team Key Results.md | 6 ++--- Writing a Roadmap.md | 8 +++---- Writing.md | 4 ++-- YouTube Channels.md | 6 ++--- 119 files changed, 345 insertions(+), 341 deletions(-) diff --git a/Agentic Engineering.md b/Agentic Engineering.md index d8d1018..54dff82 100644 --- a/Agentic Engineering.md +++ b/Agentic Engineering.md @@ -1,10 +1,10 @@ # Agentic Engineering -An agent runs tools in a loop to achieve a goal. Agents are systems where [[Artificial Intelligence Models|LLMs]] dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. Agents help decouple programming, the craft of physically typing code, from engineering, the architecture of your system, the goals, the "why" of what you're building. +An agent runs tools in a loop to achieve a [[Goals|goal]]. Agents are [[Systems|systems]] where [[Artificial Intelligence Models|LLMs]] dynamically direct their own [[Processes|processes]] and tool usage, maintaining control over how they accomplish tasks. Agents help decouple [[Programming|programming]], the craft of physically typing code, from engineering, the architecture of your system, the goals, the "why" of what you're building. - Using LLMs for coding is difficult and unintuitive, requiring significant effort to master. -- Don't delegate thinking, delegate work. -- Before coding, make a plan with the model. +- Don't delegate [[Thinking|thinking]], delegate work. +- Before coding, make a [[Planning|plan]] with the model. - You can use the same or a different model to critique the plan and iterate. If you are unsure, ask to ["give a few options before making changes"](https://steipete.me/posts/just-talk-to-it). - Progressive, iterative prototyping. [Code is cheap. Redoing work is extremely cheap](https://crawshaw.io/blog/programming-with-llms). Prioritize exploration over execution (at first). Iterate towards precision during the brainstorming phase. Start fresh once you know what and how to build it. - Failed attempts are cheap. If the plan fails and the result is bad, just delete everything and try again. @@ -26,17 +26,17 @@ An agent runs tools in a loop to achieve a goal. Agents are systems where [[Arti - Automated testing. - [Red/green TDD](https://simonwillison.net/guides/agentic-engineering-patterns/red-green-tdd/). - Planning in advance. - - Comprehensive documentation (exportable/accessible in markdown). + - Comprehensive [[Documentation|documentation]] (exportable/accessible in markdown). - Good version control habits. - - Having effective automation in place. - - A culture of code review. + - Having effective [[Automation|automation]] in place. + - A [[Culture|culture]] of code review. - Modular, reusable code. - Strong research skills. - The ability to ship to a preview environment. - An instinct for what can be outsourced (to AI vs what needs human attention). - A good (updated) sense of estimation. - Some [[programming]] rules for code design from the Unix philosophy help here: - - Modularity: Write simple parts connected by clean interfaces. + - [[Modularity]]: Write simple parts connected by clean interfaces. - Clarity: Clarity is better than cleverness. - Separation: Separate policy from mechanism. Separate interfaces from engines. - Simplicity: Design for simplicity. Add complexity only where you must. diff --git a/Analytics Engineering.md b/Analytics Engineering.md index d59d83a..9ad2338 100644 --- a/Analytics Engineering.md +++ b/Analytics Engineering.md @@ -1,6 +1,6 @@ # Analytics Engineering -- Analytics Engineering is the organization of an organization's information. +- Analytics Engineering is the organization of an [[Organizations|organization]]'s information. - [Real impact comes from continuous decision-making and implementing actions with feedback](https://news.ycombinator.com/item?id=22808006). - [More analysis might not be the best solution if you can't validate or take actions on previous analysis](https://twitter.com/ejames_c/status/1753692862548697464)! - Accept that [analytics is a mess](https://benn.substack.com/p/analytics-is-a-mess). @@ -14,7 +14,7 @@ - Treat data the same way engineers treat code. That means CI/CD, tests, frequent PRs, ... - Use [[Data Practices#Data Request Template]] when getting questions. - Analytics work can be roughly split in two buckets: - 1. Building automated [[Systems]], from metrics to [[Dashboards]], to enable self-service use cases for business users. This is what we now typically call analytics engineering. + 1. Building automated [[Systems]], from [[Metrics|metrics]] to [[Dashboards]], to enable self-service use cases for business users. This is what we now typically call analytics engineering. 2. Doing ad-hoc analyses, to answer some questions directly. - Make your modeling approach explicit (e.g. [Dimensional Modeling](https://youtu.be/lWPiSZf7-uQ)). - Modeling reality gets complex quickly. There are small nuances, special conditions, things that changed, edge cases and, of course, errors. @@ -24,8 +24,8 @@ - [[Teamwork|Collaborate with your team]] and break down complex models into reusable pieces. - Working with data is like exploring the horizon. It changes as soon as you look at it from a higher place (more data). - Find out what decisions your stakeholders need to make, repeatedly, and help with those. -- Attach a date to your team output resources ([[Dashboards]], analysis, ...) so they exist as artifacts that were true at a certain point in time. -- Reduce the areas where business logic can be injected, create "time to live" policies on last mile transforms, build a culture of standardizing + celebrating access to cross-functional codebases. +- Attach a date to your team output resources ([[Dashboards]], analysis, ...) so they exist as artifacts that were true at a certain point in [[Time|time]]. +- Reduce the areas where business logic can be injected, create "time to live" policies on last mile transforms, build a [[Culture|culture]] of standardizing + celebrating access to cross-functional codebases. - People default to writing business logic in the tool they are most comfortable with. The best way for data teams to prevent sprawling business logic is to limit last mile transforms in other tools and invite others into their tools. [The logic will be written, and if the data team gate-keeps, it will be written outside of their visibility](https://ian-macomber.medium.com/data-systems-tend-towards-production-be5a86f65561)! If a data team can educate and encourage contributions to their codebase, they invite code to be written where it most belongs. - Modern data warehouses [might need new model design paradigms](https://github.com/ActivitySchema/ActivitySchema/blob/main/2.0.md). - Good data models make good products. diff --git a/Antifragility.md b/Antifragility.md index aaea744..ca36717 100644 --- a/Antifragility.md +++ b/Antifragility.md @@ -3,7 +3,7 @@ Antifragility is a property of [[systems]] that increase in capability, resilience, or robustness as a result of stressors, shocks, volatility, noise, mistakes, faults, attacks, or failures. It's fundamentally different from the concepts of resiliency (ability to recover from failure) and robustness (ability to resist failure). - Antifragility is beyond resilience or robustness. **The resilient resists shocks and stays the same. The antifragile gets better**. -- Antifragile systems benefit from a process of trial and error. [[experimentation]] and [[learning]] from failures are essential for improving and evolving. +- Antifragile systems benefit from a [[Processes|process]] of trial and error. [[experimentation]] and [[learning]] from failures are essential for improving and evolving. - [[Evolution]] is antifragile. - In a stable system, animals won't evolve. In a volatile system, they will. - [A little bit of disorder is good](https://astralcodexten.substack.com/p/book-review-antifragile). [[Evolution]] is a system that allows volatility rather than unwisely trying to buffer against it. diff --git a/Asynchronous Communications.md b/Asynchronous Communications.md index f6428a4..ed600cf 100644 --- a/Asynchronous Communications.md +++ b/Asynchronous Communications.md @@ -1,18 +1,18 @@ # Asynchronous Communications - Having [asynchronous communication channels and making heavy use of them can have great effects on productivity](https://www.martinklepsch.org/posts/asynchronous-communication.html). -- Every question asked in an internal Slack is a policy failure. +- Every question asked in an internal [[Slack]] is a policy failure. - [Asynchronous environments allow for self-discovery without interruptions](http://web.archive.org/web/20240222195139/https://snir.dev/blog/remote-async-communication/): - You can keep in your flow without waiting for someone to give you details. - You can get into a "Deep [[Focus]]" session without context switching that allows for better [[productivity]]. - You can work whenever, since you are not dependent on anyone immediately. - You'll have written records of everything. -- [Async communication](http://web.archive.org/web/20241006094947/https://protocol.almanac.io/docs/async-work-ezPny9x7Q50QISL4UIUhB3PoURV0lgxP) takes more time but it enables better thinking. Learn to [[Asking Questions|ask better questions]] and [[Writing|write requests]]. +- [Async communication](http://web.archive.org/web/20241006094947/https://protocol.almanac.io/docs/async-work-ezPny9x7Q50QISL4UIUhB3PoURV0lgxP) takes more [[Time|time]] but it enables better [[Thinking|thinking]]. Learn to [[Asking Questions|ask better questions]] and [[Writing|write requests]]. - The 4 components of a [great asynchronous message](https://protocol-labs.gitbook.io/launchpad-curriculum/launchpad-learning-resources/protocol-labs-network/os-stewardship#sync-comms): 1. Enough information to cover all follow-up questions. 2. A deadline. When do you need a response by? How urgent is it? Which task is being blocked right now? 3. Links, images, and as much supporting material as possible that will help illustrate your thoughts. - 4. A concrete need. What do you want to get out of the communication? Approval on a task? An asset of some kind? Be clear. + 4. A concrete need. What do you want to get out of the [[Communication|communication]]? Approval on a task? An asset of some kind? Be clear. - Beware of the (cultural) Power Distance Index. The lower the PDI, the more direct communications are preferred; the higher, the more diplomatic the communications are preferred. - Asynchronous communication is efficient and inclusive, ensuring that discussions are well thought through, self-documented, and accessible to all team members, regardless of their work hours. diff --git a/Automation.md b/Automation.md index 46308ec..a80b8e6 100644 --- a/Automation.md +++ b/Automation.md @@ -3,12 +3,12 @@ - Aim for automating your whole job. [A good way is to start writing a script that only prints out the steps required to do a task](https://blog.danslimmon.com/2019/07/15/do-nothing-scripting-the-key-to-gradual-automation/). You can replace these steps one by one and do the rest manually until they're all automated. - [Not everything should be automated right away](https://xkcd.com/974/). - [Fake it, until you automate it!](https://understandlegacycode.com/blog/fake-it-until-you-automate-it/) -- Automation is used for Precision, Stability and Speed. It reduces or eliminates human error and brings stability to a system. [Automation is great when it replaces a stable, well-working manual process](https://news.ycombinator.com/item?id=30230367). +- Automation is used for Precision, Stability and Speed. It reduces or eliminates human error and brings stability to a [[Systems|system]]. [Automation is great when it replaces a stable, well-working manual process](https://news.ycombinator.com/item?id=30230367). - [Automation is putting process into code](https://queue.acm.org/detail.cfm?id=3197520): 1. Document the steps. Following a step-by-step guide is automation: you are the CPU. A great starting point is adding a [[checklist]] to PRs. 2. Create automation equivalents. Add command-line snippets to replace steps. 3. Create automation. Create a script that runs everything. - Drive standards through automation and building internal tools/scripts rather than through extensive [[documentation]]. - - Standards can help to reduce this friction but take time. Before jumping into standards processes, [consider other ways to encourage consistency](https://blog.ldodds.com/2023/09/18/consistency-before-standards/). + - Standards can help to reduce this friction but take [[Time|time]]. Before jumping into standards [[Processes|processes]], [consider other ways to encourage consistency](https://blog.ldodds.com/2023/09/18/consistency-before-standards/). - If you jump into standards, [make them open, modular, interoperable, customizable, and extensible](http://web.archive.org/web/20250327201845/https://voltrondata.com/codex/standards-over-silos). - Makefiles are a great way to document and consolidate different projects of a team. Each project should have a [`make` that runs it](https://gagor.pro/2024/02/how-i-stopped-worrying-and-loved-makefiles/), and perhaps a `make deploy` to deploy it. Language- and tool-independent! diff --git a/Blockchain.md b/Blockchain.md index c19d104..f81f476 100644 --- a/Blockchain.md +++ b/Blockchain.md @@ -1,7 +1,7 @@ # Blockchain -- [A blockchain is a decentralized [[Databases|database]]](https://www.youtube.com/watch?v=bBC-nXj3Ng4). - - A blockchain is a worse database. It is slower, requires way more storage and compute, doesn't have customer support, etc. [But has one dimension along which it is radically different. No single entity or small group of entities controls it](https://continuations.com/web3crypto-why-bother). +- [A blockchain is a decentralized database](https://www.youtube.com/watch?v=bBC-nXj3Ng4). + - A blockchain is a worse [[Databases|database]]. It is slower, requires way more storage and compute, doesn't have customer support, etc. [But has one dimension along which it is radically different. No single entity or small group of entities controls it](https://continuations.com/web3crypto-why-bother). - Blockchains solve the Byzantine Generals Problem: [How do participants in a decentralized network communicate and coordinate with each other towards some action without relying on a trusted third-party?](https://a16zcrypto.com/posts/article/crypto-glossary/). - Blockchains are "trustless". There are mechanisms in place by which all parties in the [[Systems|system]] can reach a consensus on what the canonical truth is. - Power and trust are distributed (or shared) among the network's stakeholders (e.g. developers, miners, and consumers), rather than concentrated in a single individual or entity (e.g. banks, governments, and financial institutions). @@ -12,7 +12,7 @@ - Fungible (there isn't a difference between two items. eg. storing files). All the coins are mutually interchangeable. - The resource can be provided by a lot of people. - Smart contracts can be defined as code that's replicated and executed on all the blockchain nodes. - - You can encode "incentives" in a smart contract that will be enforced by the code. This drives behavior. + - You can encode "[[Incentives|incentives]]" in a smart contract that will be enforced by the code. This drives behavior. - Smart contracts allow permissionless composability. [Composability allows anyone in a network to take existing programs and adapt or build on top of them, it unlocks completely new use cases that don't exist in our world](https://a16zcrypto.com/posts/article/how-composability-unlocks-crypto-and-everything-else/). - [Blockchains could replace networks with markets](https://twitter.com/naval/status/877467629308395521). - [They can be thought of as social and economic laboratories because they're great for experimenting with novel concepts at scale](https://x.com/binji_x/status/1891929209737470024). diff --git a/Blogging.md b/Blogging.md index 038ca31..c6741ee 100644 --- a/Blogging.md +++ b/Blogging.md @@ -20,7 +20,7 @@ 3. Previews: Give quick previews of your subtopics 4. Subtopics: Go into depth with your subtopics 5. Summaries: Recap your subtopics - 6. Conclusion: Paint an ideal vision of the future + 6. Conclusion: Paint an ideal vision of the [[Future|future]] 7. Call to action: Invite your audience to act or make a decision - For [short-form content](https://sivers.org/7): 1. Write all your thoughts on a subject. @@ -32,9 +32,9 @@ 7. Post the outline. Trash the rest. - Alternatively, try the Rift approach: 1. An inquisitive title, something that is not "the ultimate guide" but more "some notes on..." - 2. A few references. Connect the dots between some links, quotes from other sources. + 2. A few references. Connect the dots between some links, [[Quotes|quotes]] from other sources. 3. An anecdote from your own work that provides rich texture and context for what you do. - 4. Some open questions that invite people to discuss. + 4. Some [[Open Questions|open questions]] that invite people to discuss. - [Keep making your post more opinionated until it reflects your true beliefs](https://twitter.com/HamelHusain/status/1751995737095709164). Don't hedge. People want to hear what you think! - The title gets read way more than the rest, [so make it count](https://dynomight.net/titles/). **Always optimize the title and first paragraph of your post**. - Think of the title as a "classifier": it should attract those who will like the content and deter those who won't. @@ -46,10 +46,10 @@ - Make a strong claim about the content - Are almost too specific - Use unusual word combinations - - Evoke curiosity + - Evoke [[Curiosity|curiosity]] - The inverted pyramid works well for blog posts. Put the tweet-length version of the post in the title or first paragraph. Get to the point quickly, then elaborate. Readers can bail out at any point of the text and still take home most of what mattered, while the meticulous crowd can have their nitpicks addressed toward the end. - [You're not just writing for today's invisible audience](https://web.archive.org/web/20250219111210/https://andysblog.uk/why-blog-if-nobody-reads-it/). You're writing for: - - Future you. Your posts become a time capsule of your evolving mind. + - Future you. Your posts become a [[Time|time]] capsule of your evolving mind. - One right person. Maybe one day, someone stumbles across your words at exactly the right moment. And that changes something for them. - The work itself. Consistency beats virality. A hundred posts with depth will outlast a single viral hit. - If you have done something cool, or you have studied something for a long time, or you have thought something interesting, and you are writing it up, and you are at a loss how to get started, try to extract out the key phrase. What do you find yourself ranting about to people repeatedly? What does the Wikipedia entry miss that frustrates you? How would the world be different if this were not true? If you were telling a friend in a rush why you were excited to write this down, what would you say? Just say that! diff --git a/Cats.md b/Cats.md index 1eff90b..b1a9963 100644 --- a/Cats.md +++ b/Cats.md @@ -4,7 +4,7 @@ - It's recommended to let your kitty eat as much as it wills, until it reaches adulthood. They need to grow big and strong! - Buy a scratching post and other scratching toys to encourage them to scratch the right surfaces. Catnip also helps. - Slowly blinking at cats says "I'm comfortable around you" and is a show of contentment and non-hostility. -- Do things at roughly the same time every day, and stick to a schedule. They operate their entire lives off their schedule, and will be more reasonable if you are consistent with doing their stuff. +- Do things at roughly the same [[Time|time]] every day, and stick to a schedule. They operate their entire lives off their schedule, and will be more reasonable if you are consistent with doing their stuff. - Toys, buy a variety at first to see what your cat likes. - Wet food, not dry food since dry food will make their bodies work too hard and put your cat into a constant state of dehydration. - Have one of the playing sessions right before the last meal of the day. A good hunt ends with a hefty meal. After a quick grooming session, she'll [[sleep]] like an angel, allowing you to do the same. diff --git a/Checklist.md b/Checklist.md index 42c9120..47ae654 100644 --- a/Checklist.md +++ b/Checklist.md @@ -1,11 +1,11 @@ # Checklist -Checklists are an essential component of high-quality processes, e.g. in aviation, medicine or construction. [Checklists provide a useful mix of freedom and expectation, and also measure progress toward common goals](https://emilie.substack.com/p/the-swim-class-checklist). If you did a sequence of actions 3 times, make it a checklist! +Checklists are an essential component of high-quality [[Processes|processes]], e.g. in aviation, medicine or construction. [Checklists provide a useful mix of freedom and expectation, and also measure progress toward common goals](https://emilie.substack.com/p/the-swim-class-checklist). If you did a sequence of actions 3 times, make it a checklist! ## [How To Build A Checklist](http://projectmanagementhacks.com/how-to-build-a-checklist/) 1. **Identify "Stupid Mistakes" that cause failure**. Understand the most significant causes of failure. -2. **Seek Additional Input From Others**. There are other people in your organization who either do similar work or who use the results of your work. Ask these people for their ideas on the common causes of failure or what they would suggest checking. +2. **Seek Additional Input From Others**. There are other people in your organization who either do similar work or who use the results of your work. Ask these people for their [[Ideas|ideas]] on the common causes of failure or what they would suggest checking. 3. **Create Simple "Do" Steps**. Reminders to do a specific action. 4. **Test The Checklist**. Put your checklist into action. 5. **Refine the Checklist**. Refine and improve the checklist. diff --git a/Communication.md b/Communication.md index fb3971f..7f0bcf9 100644 --- a/Communication.md +++ b/Communication.md @@ -2,13 +2,13 @@ > To effectively communicate, we must realize that we are all different in the way we perceive the world and use this understanding as a guide to our communication with others — Tony Robbins -Communication is the science of transmitting knowledge to other humans. It goes by many names: debate, negotiation, [[Resolving Disagreement|discussion]], [[Talking|talking]], [[Learning|education]], [[listening]], and many more. Communication is a skill that is learnable. +Communication is the [[Science|science]] of transmitting knowledge to other humans. It goes by many names: debate, negotiation, [[Resolving Disagreement|discussion]], [[Talking|talking]], [[Learning|education]], [[listening]], and many more. Communication is a skill that is learnable. -**The person who tells the most compelling story wins**. Not the best idea. Just the story that catches people's attention and gets them to nod their heads. Tell people what they want to hear and you can be wrong indefinitely without penalty. [Crafting and telling stories is part of what makes humans humans](https://www.notboring.co/p/story-time). Stories let us coordinate across time and space. +**The person who tells the most compelling story wins**. Not the best idea. Just the story that catches people's attention and gets them to nod their heads. Tell people what they want to hear and you can be wrong indefinitely without penalty. [Crafting and telling stories is part of what makes humans humans](https://www.notboring.co/p/story-time). Stories let us [[Coordination|coordinate]] across [[Time|time]] and space. - **You cannot not communicate**. Not discussing the elephant in the room is communicating. Few things are as important to study, practice, and perfect as clear communication. - Whenever possible, communicate directly with those you're addressing rather than passing the message through intermediaries. -- Communication between [a large group](https://twitter.com/RichRogers_/status/1159872097205805056) is hard. Noise in the processes might change the message and cause conflicts. Nuance is hard to convey in groups. +- Communication between [a large group](https://twitter.com/RichRogers_/status/1159872097205805056) is hard. Noise in the [[Processes|processes]] might change the message and cause conflicts. Nuance is hard to convey in groups. - The [biggest problem with communication is the illusion that it has been accomplished](https://jabian.com/blog/the-illusion-of-communication). - Some tips to simplify communications: - Use a few bullet points to put attention on the main points you want to convey. @@ -16,7 +16,7 @@ Communication is the science of transmitting knowledge to other humans. It goes - Break up a giant nuanced block into sections. - If something is critical, make it visual. - **Beware [semantic diffusion](https://martinfowler.com/bliki/SemanticDiffusion.html)**. Popular terms lose their precise meaning as they spread through repeated misinterpretation. Complex ideas get simplified, then distorted, like a game of telephone. The more desirable a term sounds, the more likely it gets misappropriated. Keep pointing back to original definitions rather than abandoning diffused terms. -- [Use plain language for complex concepts](https://www.scientificdiscovery.dev/p/salonis-guide-to-data-visualization) — most ideas can be understood by most people if explained clearly. But keep jargon when precision matters (e.g., distinguishing mean from median); in those cases, add a brief definition alongside the technical term. +- [Use plain language for complex concepts](https://www.scientificdiscovery.dev/p/salonis-guide-to-data-visualization) — most [[Ideas|ideas]] can be understood by most people if explained clearly. But keep jargon when precision matters (e.g., distinguishing mean from median); in those cases, add a brief definition alongside the technical term. - [[Data Visualization]] can communicate faster than 1,000 words; [that power comes with responsibility](https://www.scientificdiscovery.dev/p/salonis-guide-to-data-visualization) — misleading charts spread just as easily as accurate ones. - If you want an answer, you have to [[Asking Questions|ask a question]]. People typically have a lot to say, but they'll volunteer little. diff --git a/Company Handbooks.md b/Company Handbooks.md index 0c3878d..5f53bfc 100644 --- a/Company Handbooks.md +++ b/Company Handbooks.md @@ -26,7 +26,7 @@ Handbooks are key for good [[Company Knowledge Management]]. More examples in the [Handbook of Handbooks](https://hackmd.io/@yHk1snI9T9SNpiFu2o17oA/Skh_dXNbE?type=view). -### Cooperatives +### [[Cooperatives]] - [Loomio](https://www.loomio.coop/) - [Catalyst](https://catalystcoop-handbook.readthedocs.io/en/latest/) diff --git a/Company Knowledge Management.md b/Company Knowledge Management.md index df91b55..5491526 100644 --- a/Company Knowledge Management.md +++ b/Company Knowledge Management.md @@ -25,7 +25,7 @@ There are some basic principles and [[values]] that will make maintaining and ev - Keep a source of truth and keep it up to date. When something is not relevant anymore, deprecate or delete it. All the documents should evolve. - Avoid duplicating knowledge. For each question there is one and only one answer. - Link everything together. - - The documentation should have backlinks and block references to incentivize small chunks of atomic ideas. + - The documentation should have backlinks and block references to incentivize small chunks of atomic [[Ideas|ideas]]. - When doing presentations, don't present slides, present the content of the [[Company Handbooks|company handbook]]. - Information should be easy to add (input) as well as easy to search and find (output) resulting in quick knowledge transfer between different employees. - [[Writing]] something in the wrong place is the same as not writing it. @@ -39,11 +39,11 @@ There are some basic principles and [[values]] that will make maintaining and ev - [[Documentation]] should have a common entry point to increase discovery. Each team can have its own [[processes]] on top. - At a company level, each team documents differently. To make the most of that, grow the knowledge organically and locally, not top down. Each team should have its own permissionless and open way of working and should be made public to the rest of the company. That way they can evolve the system to fit their needs. Then, there is a standard protocol to share information between teams. - The protocol serves as the team communications API. An abstraction over the inner workings of the team that is common to all the other teams. - - E.g: Each team having a homepage [README](https://tom.preston-werner.com/2010/08/23/readme-driven-development) with links to their Slack, ticket system, and processes. + - E.g: Each team having a homepage [README](https://tom.preston-werner.com/2010/08/23/readme-driven-development) with links to their [[Slack]], ticket system, and processes. - Since the protocol is shared between apps, you can build different views on top of the knowledge and tasks via APIs. - Add as much information to your tickets/issues as possible. If the tickets/issues are any good, you'll find relevant tickets with links and extra information. That makes it easy to tell what is current vs. what is 3 years old. The approach is something like Kafka (a log of everything that happened) versus a database with the current state of the world. - [Every document you write can fall into one of two categories](https://clrcrl.com/2021/11/30/fighting-the-entropy-of-knowledge.html): - - Type 1: A point-in-time document, that should lose relevance at some point. e.g. meeting notes, feature specs that get completed, [[feedback]] that gets addressed + - Type 1: A point-in-[[Time|time]] document, that should lose relevance at some point. e.g. meeting notes, feature specs that get completed, [[feedback]] that gets addressed - Type 2: A source of truth document, that should be maintained over time, e.g. "How work gets done at". May also be referred to as "evergreen" content. - Keep some basic structure, but provide a good search. - You can connect everything to Slack (search engine powered by humans). diff --git a/Cooperatives.md b/Cooperatives.md index a0d2f0f..c0d6660 100644 --- a/Cooperatives.md +++ b/Cooperatives.md @@ -1,7 +1,7 @@ # Cooperatives - A cooperative is an autonomous association of persons united voluntarily to meet their common economic, social and cultural needs and aspirations through a jointly-owned and democratically-controlled enterprise. - - Labor, capital, and governance are shared. + - Labor, capital, and [[Governance|governance]] are shared. - There are [many ways to form cooperatives](https://institute.coop/sites/default/files/resources/356%202009_Johnson%20etal_Tech%20Freelancers%20Guide%20to%20Worker%20Co-ops.pdf). - Useful when the work is not capital-intensive. E.g: [you can start curating and selling datasets, or providing consulting services with little capital (a computer and an internet connection)](https://www.youtube.com/watch?v=EjsGp5KPxx8). diff --git a/Coordination.md b/Coordination.md index 4ba1f2a..803c851 100644 --- a/Coordination.md +++ b/Coordination.md @@ -17,7 +17,7 @@ 1. Repeat interactions. 2. Look for possible win-wins. 3. Communicate clearly and [establish common context](https://gestalt.cafe/trust-infrastructure/). -- Trust is foundational to civilization, enabling coordination across time and distance. The internet has eroded traditional trust mechanisms by: +- Trust is foundational to civilization, enabling coordination across [[Time|time]] and distance. The internet has eroded traditional trust mechanisms by: - Reducing the surface area for trust-building - Expanding social contexts beyond our capacity to maintain relationships - Separating information from its contextual moorings @@ -53,12 +53,12 @@ - Make the group structure explicit, not implicit. The rules of decision-making must be open and available to everyone, and this can happen only if they are formalized. Having an established process for decision-making ensures that everyone can participate in it to some extent. - [The more you need consensus, the less work you can do](http://hintjens.com/blog:100). - A way to coordinate without trust is relying on [[Blockchain]] and [[Cryptocurrencies]]. -- Coordinating is better than who takes the resources. The more you can coordinate, the more resources you can take ([Paretotopia](https://youtu.be/1lqBra8r468)). Keep this in mind when making decisions and negotiating. +- Coordinating is better than who takes the resources. The more you can coordinate, the more resources you can take ([Paretotopia](https://youtu.be/1lqBra8r468)). Keep this in mind when [[Making Decisions|making decisions]] and negotiating. - [Ability to coordinate](https://thecompendium.cards/c/coordination-as-form-of-power) is a form of power. - [Collusion](https://vitalik.eth.limo/general/2019/04/03/collusion.html) is the dark side of coordination, where a subset of people coordinate against the interests of a broader group. - Cannot be detected by looking at actions alone, as the same actions can be legitimate or collusive depending on context and intentions. - - Requires specific countermeasures like deliberate decentralization, skin-in-the-game mechanisms, and fostering counter-coordination (e.g., forking in blockchain systems). + - Requires specific countermeasures like deliberate decentralization, skin-in-the-game mechanisms, and fostering counter-coordination (e.g., forking in blockchain [[Systems|systems]]). - Examples include price fixing between sellers, vote buying, and 51% attacks on blockchains. - Any mechanism that helps under-coordinated parties coordinate will also help already-coordinated parties over-coordinate to extract value. -- Explore [Cooperative Governance](https://nadia.xyz/voting) ideas. It offers an elegant compromise between on-chain and off-chain governance: you have a failure outcome that's enforced and "real", but the process of consensus remains firmly in the hands of people. +- Explore [Cooperative Governance](https://nadia.xyz/voting) [[Ideas|ideas]]. It offers an elegant compromise between on-chain and off-chain governance: you have a failure outcome that's enforced and "real", but the process of consensus remains firmly in the hands of people. - There are [interventions](https://www.prosocialdesign.org/) to help online spaces bring out the best in human nature online (deleting a comment that breaks a forum's rules, a reminder of the community rules, ...). This helps build trust and coordination. diff --git a/Cryptocurrencies.md b/Cryptocurrencies.md index 0423429..f1e20dc 100644 --- a/Cryptocurrencies.md +++ b/Cryptocurrencies.md @@ -4,7 +4,7 @@ - Originally, currencies were actual precious metals, like gold and silver coins. For the sake of portability, these were replaced with bank notes. Pieces of paper which entitled the bearer to a certain quantity of precious metal if they presented them to the bank. That system is known as the gold standard. - The gold standard was abandoned in the middle of the 20th century. Now we have "fiat money", which is money that has value simply because everyone agrees it has value. The biggest difference between 20 real dollars and 20 Monopoly dollars now is that you can use the real dollars to pay taxes. - Now that we have an easy way to do consensus on the internet, cryptocurrencies are simply a digital version of money. When you buy Bitcoin you're using the Blockchain to tell everyone. If you're going to spend more than you have, everyone will be able to see it! -- Proof of work is a mathematical problem that takes time to solve. Solving it proves that you've spent some time trying combinations until it worked. Proof of work creates scarcity. The blockchain knows that the miners are spending their time and resources on proving the mathematical puzzles. +- Proof of work is a mathematical problem that takes [[Time|time]] to solve. Solving it proves that you've spent some time trying combinations until it worked. Proof of work creates scarcity. The blockchain knows that the miners are spending their time and resources on proving the mathematical puzzles. - Ethereum serves as a platform (Turing complete) to create (blockchain) apps the same way Android/iOS does on mobile. - It's much more flexible than Bitcoin and that makes it riskier. - Forks in blockchain act like genetic mutations. The users will apply pressure and protocols will evolve. You can fork cryptocurrencies, you can't do that with companies and that makes blockchains more resilient. diff --git a/Culture.md b/Culture.md index c934809..f508d40 100644 --- a/Culture.md +++ b/Culture.md @@ -5,17 +5,17 @@ - It is _very_ difficult to undo bad culture — far more than it is to shape good culture. So it's best to avoid it altogether. You have to constantly be on the lookout for nascent bad behavior, in order to nip it in the bud before it becomes too bad. - A bad thing to do to the culture is introduce too much tension: - There are better battles to fight than with each other. - - It's hard to say objectively that one culture is "better" than another, but having the culture pulled in multiple directions clearly burns people out, so focus on cultural alignment in decisions and hiring. + - It's hard to say objectively that one culture is "better" than another, but having the culture pulled in multiple directions clearly burns people out, so [[Focus|focus]] on cultural alignment in decisions and hiring. - Times change, trends change, cultures change. - Foster a culture of [people who care](https://web.archive.org/web/20260426082449/https://grantslatton.com/nobody-cares). -- Culture conforms to the shape of communication. E.g: open, transparent communication fosters more trust and collaboration. +- Culture conforms to the shape of [[Communication|communication]]. E.g: open, transparent communication fosters more trust and collaboration. - You get a good culture when people want to participate in it because it is a good culture. - If a culture officially says to do one thing, but actually people do something else, then the latter is more decisive. - Culture is shaped by [[Incentives]], but then incentives are themselves implemented by people, who are guided by culture. - You want an environment [where cultures improve and compete](https://vitalik.eth.limo/general/2025/12/17/societies.html), but not on the basis of violent force, and also not exclusively on low-level forms of memetic fitness (eg. virality of individual posts on social media, moment-by-moment enjoyment and convenience), but rather on some kind of fair playing field that creates sufficient space to showcase the longer-term benefits that a thriving culture provides. -- Truly instantiating a culture with any level of depth requires not just talking about the culture's themes, but actually living them. This requires deep immersion, instantiating the culture's values and aesthetics and practices at a level that goes far beyond a few decorations and posters. +- Truly instantiating a culture with any level of depth requires not just [[Talking|talking]] about the culture's [[Themes|themes]], but actually living them. This requires deep immersion, instantiating the culture's [[Values|values]] and aesthetics and practices at a level that goes far beyond a few decorations and posters. - [Cultures are not museum pieces](https://vitalik.eth.limo/general/2025/12/17/societies.html). They are the working machinery of everyday life. Cultures are there to serve their people, not touristic onlookers appreciating their existence from far away. -- Cultural innovation works better when it arises out of a collection of habits, attitudes and goals that are shared by a particular group, and adapted to the group's needs. Over-indexing on legibility and explicit ideology often leads to problems. +- Cultural innovation works better when it arises out of a collection of [[Habits|habits]], attitudes and [[Goals|goals]] that are shared by a particular group, and adapted to the group's needs. Over-indexing on legibility and explicit ideology often leads to problems. - [Universal culture is the collection of the most competitive ideas and products](https://slatestarcodex.com/2016/07/25/how-the-west-was-won/). It's what spreads when barriers to communication fall. It's distinct from any traditional culture (including Western). Universal culture is high-entropy: it survives and spreads without protection. All other cultures are low-entropy: they survive only if someone keeps pushing energy into the system to protect them. - Universal culture has adapted to diverse environments through social atomization: everybody does their own thing, and the community exists to protect them and perform lowest common denominator functions everyone can agree on. -- Culture is [made of shared framings—ontologies](https://aethermug.com/posts/culture-is-the-mass-synchronization-of-framings) of things that are taken to exist and play a role in mental models—that arose in those same arbitrary but self-reinforcing ways. +- Culture is [made of shared framings—ontologies](https://aethermug.com/posts/culture-is-the-mass-synchronization-of-framings) of things that are taken to exist and play a role in [[Mental Models|mental models]]—that arose in those same arbitrary but self-reinforcing ways. diff --git a/Curiosity.md b/Curiosity.md index c2ddb19..3df097a 100644 --- a/Curiosity.md +++ b/Curiosity.md @@ -3,7 +3,7 @@ > Everyone you will ever meet knows something you don't. Your job is to discover what it is, and it won't be obvious. Bill Nye. - Never stop [[learning]], practice [[Thinking|rational thinking]] and internalize [[mental models]] and other concepts that will help you think more effectively. -- Knowledge is a powerful tool. The more you [feel like a noob](http://paulgraham.com/noob.html), the better. Feeling stupid now is better than feeling stupid in 10 years. +- [[Learning|Knowledge]] is a powerful tool. The more you [feel like a noob](http://paulgraham.com/noob.html), the better. Feeling stupid now is better than feeling stupid in 10 years. - [Reality has a surprising amount of detail](http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail). Knowing more about the world makes you enjoy it more. [Understanding how music is made increases the pleasure you get from music](https://youtu.be/JbVfcZxfIZo). - Real things exist in essentially infinite resolution. [Looking closer always reveals more, and it's often not what you'd expect](https://www.raptitude.com/2023/10/the-truth-is-always-made-of-details/). - Do stuff! Whatever it is you work on, is worthwhile as long as you share your learnings. In the worst case, if your [[ideas]] don't work out, the community will learn why that approach doesn't make sense. @@ -14,5 +14,5 @@ - [Practice hunting for bugs](https://radimentary.wordpress.com/2018/01/29/hammertime-day-1-bug-hunt/?utm_source=pocket_mylist)! A bug is anything in life that needs improvement. Even if something is going well, if you can imagine it going better, there's a bug. - Knowing what we really want is hard and takes effort. Explore yourself and your [[values]]. - Experiment more! It is very very easy to do what comes naturally and never deviate from that. Break the pattern. -- Consume content that is hard to produce. If the producer can spam the content, it is probably not worth your time. E.g: audiobooks vs podcasts. +- Consume content that is hard to produce. If the producer can spam the content, it is probably not worth your [[Time|time]]. E.g: audiobooks vs [[Podcasts|podcasts]]. - Having fun is efficient! Doing fun activities gives you your energy back. diff --git a/Dashboards.md b/Dashboards.md index 6a912e1..525dedd 100644 --- a/Dashboards.md +++ b/Dashboards.md @@ -8,7 +8,7 @@ - Answer three specific questions: [How, What, How](https://youtu.be/g2-dkJkZjiI)? - The data will move 1 of three directions; up, down or stay the same. Ahead of time, what will the stakeholder do in each case? Are all three answers the same? - Strive to create dashboards that are either standalone or include links to provide the relevant context. Without meaning, data is just digits. - - Ideally, push visualizations to the tools that have the context (e.g. a chart in a Slack message, a chart in a Jira ticket, a chart in HubSpot, etc). + - Ideally, push visualizations to the tools that have the context (e.g. a chart in a [[Slack]] message, a chart in a Jira ticket, a chart in HubSpot, etc). - [Make your charts professional](https://web.archive.org/web/20251230160154/https://bsky.app/profile/peck.phd/post/3lbfgjdnvy22k). [[Data Visualization]] is important. - [Add all the possible context into the dashboard](https://www.youtube.com/watch?v=Kub2bXrKmOE): - Instructions. @@ -29,7 +29,7 @@ - Be clear with your stakeholder about whether this is a one-off vs. something that should be referenced more than once. - Add an [explicit expiration date so it doesn't mold](https://mikkeldengsoe.substack.com/p/moldy-data). - [Have meetings where you check and discuss the metrics on the dashboard](https://web.archive.org/web/20240915120730/https://counting.substack.com/p/the-utility-of-an-unwatched-dashboard). This creates a powerful forcing function to look at the thing. -- Follow up and iterate on your work by marketing, improving and maintaining it into the future. +- Follow up and iterate on your work by marketing, improving and maintaining it into the [[Future|future]]. - Provide deep links in the dashboard to the underlying data or specific segmentations/filters. - Design matters a lot — if it looks bad, people won't look at it. - Learn about the tools business people use, understand them, and dive deep into their decision workflows to see where you can actually help and facilitate. @@ -39,7 +39,7 @@ - Make it obvious what the main takeaway is. - Follow the Z pattern. - Provide context. -- Include additional highlight numbers, such as last month's total revenue vs. the planned revenue and the YoY growth. Plot against goals if possible. Have a reasonable goal, and plot your numbers against it! +- Include additional highlight numbers, such as last month's total revenue vs. the planned revenue and the YoY growth. Plot against [[Goals|goals]] if possible. Have a reasonable goal, and plot your numbers against it! - [Don't use cumulative charts. They are always going up and hide the underlying trend](https://youtu.be/q94VJ3KToK8?list=PLPnZfvKID1Sje5jWxt-4CSZD7bUI4gSPS). - Instead, [count number of users doing "valuable things" per day](https://x.com/jeff_weinstein/status/2034757055769350395). @@ -55,7 +55,7 @@ The value is that now discussions are happening about the data. - Dashboards were created to monitor and not to derive insights. - Dashboards report on current status. Users don't act on status. They act on change in status. - Dashboards (lines and rectangles) are useful to notice if something goes wrong. - - **It's usually not possible to generate meaningful insight simply by looking at line charts in a dashboard**. A chart alone cannot possibly convey everything, and that kind of thinking inhibits our ability to influence the business with our work. + - **It's usually not possible to generate meaningful insight simply by looking at line charts in a dashboard**. A chart alone cannot possibly convey everything, and that kind of [[Thinking|thinking]] inhibits our ability to influence the business with our work. - Especially, if [there are 10 unrelated charts in the same dashboard](http://web.archive.org/web/20250718040500/http://deathofdashboards.com/). - Building a dashboard requires gathering a lot of context. Once built, only a few users aware of all the context can really use it in the proper way. - [They can serve endless needs, but in doing so, rarely do they serve _particular_ needs perfectly](https://win.hyperquery.ai/p/analysis-or-dashboard). diff --git a/Data Culture.md b/Data Culture.md index b942d63..4f23581 100644 --- a/Data Culture.md +++ b/Data Culture.md @@ -1,8 +1,8 @@ # Data Culture -- The data team needs to be focused on delivering insights and supporting decisions. The outcomes of the data team are _decisions_ and a _shared context across the organization_ that makes coordination easier. +- The data team needs to be focused on delivering insights and supporting decisions. The outcomes of the data team are _decisions_ and a _shared context across the [[Organizations|organization]]_ that makes [[Coordination|coordination]] easier. - Your goal as a data professional is to facilitate [[Making Decisions|decision making]] and [help surface/investigate the performance of a business](https://sqlpatterns.com/p/delivering-value-as-a-data-team) (e.g. [operational](https://twitter.com/ergestx/status/1731324299590479989)). - - Learning to drive decisions quickly, a bias to action, is a critical competency for an analyst. Every skill you learn – [[communication]], [[writing]], [[experimentation]], [[Metrics|metric design]] – supports this. + - [[Learning]] to drive decisions quickly, a bias to action, is a critical competency for an analyst. Every skill you learn – [[communication]], [[writing]], [[experimentation]], [[Metrics|metric design]] – supports this. - [If analysis is not actionable, it does not really matter](https://twitter.com/decisionleader/status/1661041373783441408). Analysis must drive to action. [Clear results won't spur action themselves](https://www.linkedin.com/posts/eric-weber-060397b7_data-analytics-machinelearning-activity-6675746028144205824-CQxW/). The organization needs to be ready to pivot when something isn't working. - [Data doesn't make decisions, people do](https://twitter.com/teej_m/status/1765475939084029956). - [Data's impact is tough to measure — it doesn't always translate to value](https://dfrieds.com/articles/data-science-reality-vs-expectations.html) @@ -14,7 +14,7 @@ - Run _[Purpose Meetings](https://www.avo.app/blog/tracking-the-right-product-metrics)_ or [Business Metrics Review](https://youtu.be/nlMn572Dabc). - Purpose Meetings are 30-minute meetings in which stakeholders, engineers and data align on the goal of a release and what is the best way to evaluate the impact and understand its success. Align on the goal, commit on metrics and design the data. - Business Metrics Review is a 30- to 60-minute meeting to chat and explore key metrics and teach how to think with data. - - You don't hit a quantitative goal by focusing on the goal. You hit a quantitative goal by focusing on the process. + - You don't hit a quantitative goal by focusing on the goal. You hit a quantitative goal by focusing on the [[Processes|process]]. - Business Reviews are one of the best ways to get people to think about data. - Value of clear goals and expectations. Validate what you think your job is with your manager and stakeholders, repeatedly. - [Weekly Business Review meetings are a process control tool](https://commoncog.com/the-amazon-weekly-business-review/). A tool designed to uncover and disseminate the causal structure of a business. @@ -28,7 +28,7 @@ - Warning. Overreacting to random noise week-to-week wastes energy. - Don't report numbers without generating decisions, next steps, or deeper investigations. - Leaders prune, refine, or add new input metrics. -- [While the output of your team is what you want to maximize, you'll need some indicators that will help guide you day-to-day](http://web.archive.org/web/20240303143022/https://data-columns.hightouch.io/your-first-60-days-as-a-first-data-hire-weeks-3-4/). Decide what's important to you (test coverage, documentation missing, queries run, models created, ...), and generate some internal reports for yourself. +- [While the output of your team is what you want to maximize, you'll need some indicators that will help guide you day-to-day](http://web.archive.org/web/20240303143022/https://data-columns.hightouch.io/your-first-60-days-as-a-first-data-hire-weeks-3-4/). Decide what's important to you (test coverage, [[Documentation|documentation]] missing, queries run, models created, ...), and generate some internal reports for yourself. - [Data teams should be a part of the business conversations from the beginning](https://cultivating-algos.stitchfix.com/). Get the data team involved early, have open discussions with them about the existing work, and how to prioritize new work against the existing backlog. Don't accept new work without addressing the existing bottlenecks, and don't accept new work without requirements. **Organizational [[politics]] matter way more than any data methods or technical knowledge**. The hard bit about becoming data driven in business isn't the technical bits. It's the political bits. - Including data people in meetings causes happy accidents! - The layout of the organization impacts the time it takes for information to propagate and adds losses. @@ -70,7 +70,7 @@ - [So much of data work is about accumulating little bits of knowledge and building a shared context in your org so that it's possible to have the big, earth shattering revelations we all wish we could drive on a predictable schedule](https://twitter.com/imightbemary/status/1536368160961572864). - A big purpose of data is knowledge. Knowledge is **"theories or models that allow you to predict the outcomes of your business actions"**. Insights may originate from data but are confirmed through actions. - You won't have the best allocation of resources in a reactive team. Data teams need extra [[slack]]. [Balance user requests with actual needs](https://scientistemily.substack.com/p/product-management-skills-for-data). -- Do weekly recaps in Slack to highlight key items, company-wide progress toward north-stars, improvements in certain areas, new customer highlights. All positive and fun stuff. +- Do weekly recaps in [[Slack]] to highlight key items, company-wide progress toward north-stars, improvements in certain areas, new customer highlights. All positive and fun stuff. - How can we measure the data team impact? - Making a [[Writing a Roadmap|roadmap]] can help you tell if you are hitting milestone deadlines or letting them slip. - Embedded data team members need to help other teams build their roadmap too. @@ -82,7 +82,7 @@ - If a definition or business logic is changed, backfills should occur automatically and data remains up-to-date. - [Organizations have _too much_ data. Without better ways of organizing it, large volumes of data are more overwhelming than useful.](https://towardsdatascience.com/good-data-citizenship-doesnt-work-265f13a37fa5) - Use the questions people are asking to find data _hotspots_ and focus our energy on those. That means some corners of your data will be messy, and some concepts will go undocumented. Data is perennially broken and messy. **Embrace the mess**. - - Get excited when people ask questions. Embrace confusion and curiosity. Offer help. Be friendly. + - Get excited when people ask questions. Embrace confusion and [[Curiosity|curiosity]]. Offer help. Be friendly. - [Reality is complex and multidimensional and often difficult to comprehend](https://mobile.twitter.com/rahulj51/status/1485429967131639808). - [Document data when it's generated](https://davidsj.substack.com/p/the-data-chasm). Make it part of the process of adding a new event, table, or a replication job, when the change is already top of mind. If possible, embed it in the development process, and pester people when they don't include the necessary updates. This shifts the burden of documentation upstream, making it part of the development cycle. - To align stakeholders' incentives with the data team, stakeholders should show their impact through data. This forces stakeholders to [[Product Analytics|plan tracking]] and think about metrics. @@ -111,7 +111,7 @@ - Most of the work done in data is in an effort to **reduce entropy** — Model data to remove inaccuracies, turn commonly asked questions into self-serve reports, and funnel ad-hoc questions into a formalized request process. This kind of attitude is the nature of data practitioners. In the case of driving decisions with data, **embrace the chaos**. - Data doesn't so much drift towards entropy, **but sprints at it**. - [Navigating the chaos to arrive at a trustworthy recommendation is one of the most important jobs to be done.](https://roundup.getdbt.com/p/iterating-on-your-data-team). Decisions usually need to be taken faster and data analysts are [not invited to the table early enough](https://petrjanda.substack.com/p/bring-data-analyst-to-the-table). Again, be lean and iterate. - - Data is _not_ a "set it and forget it" kind of activity. Your dashboard _will_ get stale in less than six months. Your key metrics _will_ eventually have bad data in them. That machine learning model you spent all of last quarter developing _will_ **[drift](http://web.archive.org/web/20221010060836/https://towardsdatascience.com/model-drift-in-machine-learning-models-8f7e7413b563)** from its original fit. The environment in which your business operates is constantly changing, and so will the product or service that your business delivers. As a result, what is knowable about your business, about your product or service, is constantly changing too. And fast. + - Data is _not_ a "set it and forget it" kind of activity. Your dashboard _will_ get stale in less than six months. Your key metrics _will_ eventually have bad data in them. That [[Machine Learning|machine learning]] model you spent all of last quarter developing _will_ **[drift](http://web.archive.org/web/20221010060836/https://towardsdatascience.com/model-drift-in-machine-learning-models-8f7e7413b563)** from its original fit. The environment in which your business operates is constantly changing, and so will the product or service that your business delivers. As a result, what is knowable about your business, about your product or service, is constantly changing too. And fast. - No model is perfect. Reality has many more dimensions. Models exist **within a context**. - [Have regular cleanups and audits to keep data in check](http://web.archive.org/web/20231004070856/https://www.avo.app/blog/data-literacy-why-people-dont-trust-data-tips-from-patreons-dir-of-data-science). They are crucial to keeping your data trust up to par. [Schedule time to delete stuff](https://twitter.com/EdDaWord/status/1532148425487097857). - We're moving from software consumers to data consumers. Data and BI will become more and more federated (you get data insights on your JIRA card without having to leave JIRA). diff --git a/Data Engineering.md b/Data Engineering.md index 148dd5b..fc6e4c8 100644 --- a/Data Engineering.md +++ b/Data Engineering.md @@ -8,7 +8,7 @@ - If it can be solved with SQL, stick to SQL. - SQL will be the abstraction layer in streaming too so you don't have to care about incremental materialization or timely dataflows. - A [consistent pattern](https://www.startdataengineering.com/post/design-patterns/) across your data pipelines helps devs communicate easily and understand code better. -- Data Engineering can learn from decentralized systems ideas like content-addressed data, immutability, and [[Idempotence]]. +- Data Engineering can learn from decentralized systems [[Ideas|ideas]] like content-addressed data, immutability, and [[Idempotence]]. - [Schemas aren't eliminated by using a "schemaless" data store](https://ludic.mataroa.blog/blog/flexible-schemas-are-the-mindkiller/) (like a NoSQL database). They're just pushed to the reading layer. - [Bring compute to data instead of data to compute](https://youtu.be/S0mviKhVmBI)! @@ -16,7 +16,7 @@ Data Pipelines are a set of actions that extract data, transform it, and then load the final data somewhere. As any [distributed system](https://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/), they're tricky to work with. These are some great principles to keep in mind as [production data engineering is mostly computer science](https://web.archive.org/web/20221218002759/https://towardsdatascience.com/lessons-from-a-year-in-the-data-science-trenches-f06efa6355fd). -Systems tend towards production and data pipelines aren't an exception. Valuable data work and outputs end up being consumed in use cases that are increasingly more important / production grade. +[[Systems]] tend towards production and data pipelines aren't an exception. Valuable data work and outputs end up being consumed in use cases that are increasingly more important / production grade. ### Basic Principles @@ -24,7 +24,7 @@ Systems tend towards production and data pipelines aren't an exception. Valuable - **Reliability**: Errors in the pipelines can be recovered. Pipelines are monitored and tested. Data is saved in each step (storage is cheap) so it can be used later if needed. For example, adding a new column to a table can be done by extracting the column from the intermediary data without having to query the data source. It is better to support 1 feature that works reliably and has a great UX than 2 that are unreliable or hard to use. One solid step is better than 2 finicky ones. - **[[Modularity]]**: Steps are independent, declarative, and [[Idempotence|idempotent]]. This makes pipelines composable. - **Consistency**: Same conventions and design patterns across pipelines. If a failure is actionable for the user, clearly let them know what they can do. Schema on write as there is always a schema. -- **Efficiency**: Low event latency when needed. Easy to scale up and down. A user should not be able to configure something that will not work. Don't mix heterogeneous workloads under the same tooling (e.g: big data warehouses doing simple queries 95% of their time and 1 big batch once a day). +- **Efficiency**: Low event latency when needed. Easy to scale up and down. A user should not be able to configure something that will not work. Don't mix heterogeneous workloads under the same tooling (e.g: big data warehouses doing simple queries 95% of their [[Time|time]] and 1 big batch once a day). - **Flexibility**: Steps change to conform to data points. Changes don't stop the pipeline or lose data. Fail fast and upstream. ### Data Flow @@ -43,7 +43,7 @@ graph LR; - Data types don't change in the same column. - Columns are either deleted or added but never renamed. - Create a few extra columns like `processed_at` or `schema_version`. -- Generate stats to provide the operator with feedback. +- Generate stats to provide the operator with [[Feedback|feedback]]. - Data coming from pipelines should be easily reproducible. If you want to re-run a process, you should ensure that it will always produce the same result. This can be achieved by enforcing the [Functional Data Engineering Paradigm](https://medium.com/@maximebeauchemin/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a). - [Event Sourcing is a great pattern when implementing a new system since it couples state with business logic](https://youtu.be/XxKnTusccUM). - State is a projection of history. Keep the history and reconstruct the state! diff --git a/Data IDE.md b/Data IDE.md index 7d9f99e..1f9ad4e 100644 --- a/Data IDE.md +++ b/Data IDE.md @@ -1,21 +1,21 @@ # Data IDE -After playing with [Rill Developer](https://github.com/rilldata/rill-developer), DuckDB, Vega, WASM, [Rath](https://rath.kanaries.net/), and other modern Data IDEs, I think we have all the pieces for an awesome web-based BI/Data exploration tool. Some of the features it could have: +After playing with [Rill Developer](https://github.com/rilldata/rill-developer), DuckDB, Vega, WASM, [Rath](https://rath.kanaries.net/), and other modern Data IDEs, I think we have all the pieces for an awesome [[Web Based Tools|web-based]] BI/Data exploration tool. Some of the features it could have: - Let me add local and remote datasets. Not just one as I'd like to join them later. - Let me plot it using Vega-Lite. Guide me through alternatives like [Vega's Voyager2](https://vega.github.io/voyager2/) does. - Might be as simple as surfacing Observable Plot with DuckDB WASM... -- Use LLMs to improve the datasets and offer next steps: +- Use [[Artificial Intelligence Models|LLMs]] to improve the datasets and offer next steps: - Get suggested transformations for certain columns. If it detects a date, extract day of the week. If it detects a string, `lower()` it... - - Get suggested plots. Given that it'll know both the column names and the types. Should be possible to create a prompt that returns some plot ideas and another that takes that and writes the Vega-Lite code to make it work. + - Get suggested plots. Given that it'll know both the column names and the types. Should be possible to create a prompt that returns some plot [[Ideas|ideas]] and another that takes that and writes the Vega-Lite code to make it work. - Make it easy to query the data via natural language. - Let me transform them with SQL ([DuckDB](https://duckdb.org/)) and Python ([JupyterLite](https://jupyterlite.readthedocs.io/en/latest/)). Similar to [Neptyne](http://web.archive.org/web/20250306181451/https://www.neptyne.com/) but in the browser (WASM). - Let me save the plots in a separate space and give me a shareable URL-encoded link. - Local datasets could be shared using something like [Magic Wormhole](https://github.com/magic-wormhole/magic-wormhole) or a temporary storage service. -- Let me grab the state of the app (YAML/JSON), version control it, and generate static (to publish in GitHub Pages) and dynamic (hosted somewhere) dashboards from it. +- Let me grab the state of the app (YAML/JSON), version control it, and generate static (to publish in GitHub Pages) and dynamic (hosted somewhere) [[Dashboards|dashboards]] from it. - Similar to [evidence.dev](https://evidence.dev/) or [portal.js](https://portaljs.org/). - It could also have "smart" data checks. Similar to [deepchecks](https://github.com/deepchecks/deepchecks) alerting about anomalies, outliers, noisy variables, ... -- Given a large amount of Open Data. It could offer a way for people to upload their datasets [and get them augmented](http://web.archive.org/web/20250108164736/https://subsets.io/). +- Given a large amount of [[Open Data]]. It could offer a way for people to upload their datasets [and get them augmented](http://web.archive.org/web/20250108164736/https://subsets.io/). - E.g: Upload a CSV with year and country and the tool could suggest GDP per capita or population. Could be an awesome front end to explore Open Data. diff --git a/Data Package Manager.md b/Data Package Manager.md index c44e347..af969f6 100644 --- a/Data Package Manager.md +++ b/Data Package Manager.md @@ -1,11 +1,11 @@ # Data Package Manager Package managers have been hailed among the most important innovations Linux brought to the computing industry. -Data is not code, but the activities of both publishers and users of datasets resemble those of authors and users of software packages. Package managers provide frictionless access to code, a universal catalogue, standardized metadata, versioning, and a social/reputation layer (e.g: installs). +Data is not [[Programming|code]], but the activities of both publishers and users of datasets resemble those of authors and users of software packages. Package managers provide frictionless access to code, a universal catalogue, standardized metadata, versioning, and a social/reputation layer (e.g: installs). ## Principles -- **Distribution**. Decentralized. No central authority. Can work in closed and private networks. Cache/CDN friendly. +- **Distribution**. [[Decentralized Protocols|Decentralized]]. No central authority. Can work in closed and [[Privacy|private]] networks. Cache/CDN friendly. - A data package is a URI ([like in Deno](https://deno.land/manual@v1.31.2/examples/manage_dependencies)). You can import from a URL (`data add example.com/dataset.yml` or `data add example.com/hub_curated_datasets.yml`). - As Rufus Pollock puts it, Keep it as simple as possible. Store the table location and schema and get me the data on the hard disk (or my browser) fast. - [Bootstrap a package registry](https://antonz.org/writing-package-manager/). E.g: a GitHub repository with lots of known `datapackages` that acts as a fallback and quick way to get started with the tool (`data list` returns a bunch of known open datasets and integrates with platforms like Hugging Face). @@ -17,14 +17,14 @@ Data is not code, but the activities of both publishers and users of datasets re - For example, integrate all [Hugging Face datasets](https://huggingface.co/docs/datasets/index) by making a scheduled job that builds a Frictionless Catalog (bunch of `datapackage.yml`s pointing to their parquet files). - [Expose a JSON-LD so Google Dataset Search can index it](https://developers.google.com/search/docs/appearance/structured-data/dataset). - Data assets should be Findable, Accessible, Interoperable, and Reproducible. [FAIR](https://www.go-fair.org/fair-principles/). - - Finding the right dataset to answer a question is difficult. Good metadata search is essential. - - Have one profile per user/organization with multiple data products (e.g: how GitHub has one profile per user with multiple repositories, or Hugging Face has one profile per user with multiple datasets). + - Finding the right [[Open Data|dataset]] to answer a question is difficult. Good metadata search is essential. + - Have one profile per user/[[Organizations|organization]] with multiple data products (e.g: how GitHub has one profile per user with multiple repositories, or Hugging Face has one profile per user with multiple datasets). - **Formatting**. Datasets are saved and exposed in multiple formats (CSV, Parquet, ...). Could be done in the backend, or in the client when pulling data (WASM). The package manager should be **format- and storage-agnostic**. Give me the dataset with ID `xyz` as a CSV in this folder. -- **Social**. Allow users, organizations, stars, citations, attaching default visualizations (d3, [Vega](https://vega.github.io/), [Vegafusion](https://github.com/vegafusion/vegafusion/), and others), ... +- **Social**. Allow users, organizations, stars, citations, attaching default [[Data Visualization|visualizations]] (d3, [Vega](https://vega.github.io/), [Vegafusion](https://github.com/vegafusion/vegafusion/), and others), ... - Importing datasets. Make it possible to `data fork user/data`, improve something, and publish the resulting dataset back (via something like a PR). - Have issues and discussions close to the dataset. - Support for different licensing models beyond "fully open" (e.g., attribution, non-commercial). - - Linking data to other data makes all the data more valuable. + - [[Knowledge Graphs|Linking data to other data]] makes all the data more valuable. - Default protocol or social network for sharing datasets (similar to how email or RSS feeds work). - **Extensible**. Users could extend the package resource (e.g: [Time Series Tabular Package inherits from Tabular Package](https://specs.frictionlessdata.io/tabular-data-package/)) and add better support for more specific kinds of data (geographical). - Build integrations to ingest and publish data in other hubs (e.g: CKAN, HuggingFace, ...). @@ -32,7 +32,7 @@ Data is not code, but the activities of both publishers and users of datasets re ## Storage and Serialization -- **Permanence**. Each [version](http://web.archive.org/web/20211209080049/http://tech.datopian.com/versioning/) should be permanent and accessible (look at `git`, `IPFS`, `dolt`, ...). +- **Permanence**. Each [version](http://web.archive.org/web/20211209080049/http://tech.datopian.com/versioning/) should be permanent and accessible (look at [[Git|`git`]], [[IPFS|`IPFS`]], `dolt`, ...). - **Versioning**. Should be able to manage _diffs_ and _incremental changes_ in a smart way. E.g: only storing the new added rows or updated columns. - Should allow [automated harvesting of new data](http://web.archive.org/web/20211209080017/http://tech.datopian.com/harvesting/) with sensors (external functions) or scheduled jobs. - Each version is referenced by a hash. Git style. @@ -42,14 +42,14 @@ Data is not code, but the activities of both publishers and users of datasets re - Tabular data could be partitioned to make it easier for future retrieval. - Implement [common column standards](https://docs.google.com/document/d/1iTl7YWfTAzp8zNXRs01RAIWCP-pRJwQfDg8lsD0TDCM/edit?tab=t.0) (e.g: `country_code`, `date`, ...) that allow for data validation and standardization across datasets. These also help with interoperability / integration. - We can't cooperate on global issues if we can't agree on how to refer to things! - - Use standardized identifiers and aggregation levels to ensure data quality and compatibility. + - Use standardized identifiers and aggregation levels to ensure [[Data Quality|data quality]] and compatibility. - **Immutability**. Never remove historical data. Data should be append-only. - Many public data sources issue restatements or revisions. The protocol should be able to handle this. - [Higher resolution is more valuable than lower resolution](https://www.linkedin.com/pulse/re-framing-open-data-john-weigelt/). Publish immutable data and then compute the lower resolution data from it. - - Similar to how `git` deals with it. You _could_ force the deletion of something in case that's needed, but that's not the default behavior. -- **Flexible**. Allow arbitrary backends. Both centralized ([S3](https://twitter.com/quiltdata/status/1569447878212591618), GCS, ...) and decentralized (IPFS, Hypercore, Torrent, ...) layers. + - Similar to how [[Git|`git`]] deals with it. You _could_ force the deletion of something in case that's needed, but that's not the default behavior. +- **Flexible**. Allow arbitrary backends. Both centralized ([S3](https://twitter.com/quiltdata/status/1569447878212591618), GCS, ...) and decentralized ([[IPFS]], Hypercore, Torrent, ...) layers. - As agnostic as possible, supporting many types of data; tables, geospatial, images, ... - - Can all datasets be represented as tabular datasets? This will enable running SQL (`select`, `group by`, `joins`) on top of them which might be the easier way to start collaborating. + - Can all datasets be represented as tabular datasets? This will enable running SQL (`select`, `group by`, `joins`) on top of them which might be the easier way to start [[Coordination|collaborating]]. - A dataset could have different formats derived from a common one. Build converters between formats relying on the Apache Arrow in-memory standard format. This is similar to how Pandoc and LLVM work! The protocol could do the transformation (e.g: CSV to Parquet, JSON to Arrow, ...) automagically and run some checks at the data level to verify they contain the same information. - Datasets could be tagged from a library of types (e.g: `ip-address`) and [conversion functions](https://github.com/jbenet/transformer) (`ip-to-country`). Given that the representation is common (Arrow), the transformations could be written in multiple languages. @@ -75,8 +75,8 @@ Data is not code, but the activities of both publishers and users of datasets re ## Consumption - **Accessible**. Datasets are **files**. Datasets are static assets living somewhere. Don't get in the middle with libraries, gated databases or [obscure licenses](https://www.heltweg.org/posts/how-to-make-sure-no-one-cares-about-your-open-data/). Keep in mind right to exit and decentralization. -- **Documentation**. Surface derived work (e.g: reports, other datasets, ...) and not only the raw data with minimal metadata. -- **Embedded Visualizations**. Know what's in there before downloading it. +- **[[Documentation]]**. Surface derived work (e.g: reports, other datasets, ...) and not only the raw data with minimal metadata. +- **Embedded [[Data Visualization|Visualizations]]**. Know what's in there before downloading it. - **Sane Defaults**. Suggest basic charts (bars, lines, time series, clustering). Multiple [views](http://web.archive.org/web/20211209065537/http://tech.datopian.com/views/). - **Exploratory**. Allow drill-downs and customization. Offer a [simple way](https://lite.datasette.io/) for people to query/explore the data. - **Dynamic**. Use only the data you need. No need to pull 150GB. @@ -87,7 +87,7 @@ Data is not code, but the activities of both publishers and users of datasets re - Make sure the datasets are there for the long run. This might take different forms (using a domain name, IPFS, ...). - **Subscribable**. Datasets are not just derived from upstream data, they should actually be updated every time the upstream data is updated (e.g: recomputing a metric of wildfire risk whenever new satellite imagery becomes available). - **Dual Role Support**. Recognize that data providers are often data users and vice versa, similar to how GitHub understands that code writers are also code consumers. Design interfaces and tools that support this dual role effectively. -- **Data Product**. A data product is a collection of data created explicitly to be used by someone else (dataset, dashboard, report, benchmark, ...). Incentivize data products as they are the atomic unit of consumption for data. +- **Data Product**. A data product is a collection of data created explicitly to be used by someone else (dataset, [[Dashboards|dashboard]], report, benchmark, ...). [[Incentives|Incentivize]] data products as they are the atomic unit of consumption for data. - Data is an _experience good_. Its true value often remains uncertain until after it has been used and analyzed. This characteristic creates information asymmetries between data collectors (like local communities) and potential buyers or users, making quality assurance and standardization crucial for building digital trust in data markets. ## Simple Package Manager Design diff --git a/Data Practices.md b/Data Practices.md index 112941f..ab858d1 100644 --- a/Data Practices.md +++ b/Data Practices.md @@ -17,7 +17,7 @@ Inspired by [Caitlin Hudon's Intake Form for Data Requests](https://caitlinhudon ### Data Request Form - What kind of deliverable would be most helpful for your request? -- Will you need this data again in the future? +- Will you need this data again in the [[Future|future]]? - Which stats will this change or which action will be taken based on this data and by whom? - How can we measure our progress/success for each step? - What happens if we don't hit the target? @@ -28,7 +28,7 @@ Inspired by [Caitlin Hudon's Intake Form for Data Requests](https://caitlinhudon - [What are we trying to improve](https://twitter.com/ergestx/status/1758538405695086829)? - **Request Description** - Do you know where the related data is? -- What time period do you care about? +- What [[Time|time]] period do you care about? - Can this analysis be done in our current BI tools? - If yes, do you have a starting link? - If no, can [[Reverse ETL]] help? @@ -40,14 +40,14 @@ Inspired by [Caitlin Hudon's Intake Form for Data Requests](https://caitlinhudon - Who will see this deliverable? - How will this deliver business value within 90 days? -## Weekly Metrics Review +## Weekly [[Metrics]] Review - Beware of the question "Why did metric X go down last week?" It's frustrating to not know. But the reality is that there's often random spikes of usage, seasonality, and other external factors. If you chase every dip, you'll make up narratives that aren't true. - [You'll find 95% of metrics fall into "business as usual"](https://twitter.com/teej_m/status/1293637500703924225) and don't need to be discussed. When you report the weekly metrics, split into: 1. Notable trends. 2. Watching close. 3. Business as usual. -- Make people talk and explore their metrics. +- Make people talk and explore their [[Metrics|metrics]]. ## Data Platform UX Survey @@ -76,7 +76,7 @@ We need to know what works, what doesn't and what people are using. The aim is to answer the following questions each time: -1. **What am I looking at?** A **short-but-informative title** can tell people immediately what data is the focus of the insight. +1. **What am I looking at?** A **short-but-informative title** can tell people immediately what data is the [[Focus|focus]] of the insight. 2. **What should I learn from this?** or, Why should I care? Include the **most useful information,** and/or a **clear takeaway**. For folks who only have a few seconds to scan the message, it should be **easy to spot** the **most valuable** bit of the insight, the **reason** this exploration was considered worth sharing. 3. **What caught my eye?** Share a chart or a related resource! 4. **What if I want to know more?** A **link to additional information** can be valuable for people who have time for more than a quick scan and want to understand how you developed the insight, or do some of their own related exploration. @@ -84,7 +84,7 @@ The aim is to answer the following questions each time: 6. **What if posting this prompts a whole bunch of follow-up questions, or exposes incorrect assumptions?** If you have hit on something that's interesting to a lot of people there likely will be questions that spin off, new ways to slice the data you're looking at, or assumptions you have made that need to be corrected. - Apply [[Data Visualization]] principles so shared charts stay clear when they travel. -### Slack Template +### [[Slack]] Template ```md ![Chart]() diff --git a/Data Quality.md b/Data Quality.md index 5e18639..9a3d3a0 100644 --- a/Data Quality.md +++ b/Data Quality.md @@ -1,12 +1,12 @@ # Data Quality -- One of the biggest misconceptions about data is that **it is correct** or it will be at some point. Data quality will always be an issue, defining metrics for data quality helps you know where you stand, improve over time, and align with your team. +- One of the biggest misconceptions about data is that **it is correct** or it will be at some point. Data quality will always be an issue, defining [[Metrics|metrics]] for data quality helps you know where you stand, improve over [[Time|time]], and align with your team. - Log data quality issues so you can act on them. - [We live in a messy world and as a result the data we collect and represent it are always going to be messy](https://twitter.com/KostasPardalis/status/1641122521775849497). - Data quality [is a measurement of the value of data to the business, meaning it is dependent on the task trying to be accomplished](https://tayloramurphy.substack.com/p/data-quality). High-quality data in one context can be low in another. - [Data issues are fractal](https://web.archive.org/web/20241212162417/https://www.speedwins.tech/posts/some-words-with-nuria-ruiz). - [There are many common data quality problems already figured out](https://b-greve.gitbook.io/beginners-guide-to-clean-data/). -- Incremental data quality improvements can lead to measurable value back to the business. That's what you should focus on and this is what's going to help you with convincing people to care about data quality. +- Incremental data quality improvements can lead to measurable value back to the business. That's what you should [[Focus|focus]] on and this is what's going to help you with convincing people to care about data quality. - Data quality is a team sport. Data is not the concern of just the DEs, ML Engs, Analysts etc. Data quality starts from the front-end. Did you check the JSON you sent on Rudderstack before merging that instrumentation PR? - A table might be good enough for one consumer but not for the other consumer. - [[Data Culture|Data is a product]] and that means that you can do postmortems (5 whys) and incident reports on data quality issues. @@ -20,7 +20,7 @@ - **Accuracy**: Does the data accurately describe the real world? - **Completeness**: How completely does the data describe the real world? - **Consistency**: Is the data internally consistent? - - **Privacy and security**: Is data being used in accordance with the intended level of privacy and secured against undesired access? + - **[[Privacy]] and security**: Is data being used in accordance with the intended level of privacy and secured against undesired access? - **Freshness**: Does the data describe the real world right now? - **Reliability**: Is the data regarded as true and credible by the stakeholders? Does it contain duplicates? - **Usability**: Can the data be accessed and understood in a low-friction manner? @@ -29,4 +29,4 @@ - **Start from the use-cases**: What are the most important use cases of data within your organization? - **Identify the pain points**: Within those use cases, what data quality issues have caused the most trouble recently? - **Connect to data quality dimensions** and **describe how to measure them as metrics**: Of the causes of recent trouble, how do they relate to the previous data quality dimensions? - - **Make metrics digestible and actionable**: Lastly, how can we make those metrics useful? + - **Make [[Metrics|metrics]] digestible and actionable**: Lastly, how can we make those metrics useful? diff --git a/Databases.md b/Databases.md index 7251fc7..6eaf0d9 100644 --- a/Databases.md +++ b/Databases.md @@ -2,7 +2,7 @@ - As in any other place, stick to a naming convention to keep things sane. - Storage is cheap, make use of it. -- Follow a SQL style guide. +- Follow a [[Data Practices|SQL style guide]]. - [Kickstarter's SQL Style Guide](https://gist.github.com/fredbenenson/7bb92718e19138c20591) - [Meadmaker's SQL Style Guide](https://github.com/meadmaker/sql-style-guide) - [SQL Style Guide](https://www.sqlstyle.guide/) diff --git a/Datathons.md b/Datathons.md index bb7d9c2..6ddc112 100644 --- a/Datathons.md +++ b/Datathons.md @@ -2,6 +2,6 @@ > Tools and resources for datathons moved to [the datathon kit](https://github.com/davidgasquez/datathon-kit). -- Datathons are a great way to learn new skills and get experience in a short amount of time. +- Datathons are a great way to [[Learning|learn]] new skills and get experience in a short amount of [[Time|time]]. - Doing datathons helps you keep up with the latest trends and technologies. - Have a [Datathon Kit](https://github.com/davidgasquez/datathon-kit) ready to go to get started quickly. diff --git a/Decentralized Autonomous Organizations.md b/Decentralized Autonomous Organizations.md index 3e8be1d..9b3a460 100644 --- a/Decentralized Autonomous Organizations.md +++ b/Decentralized Autonomous Organizations.md @@ -1,7 +1,7 @@ # Decentralized Autonomous Organizations - A Decentralized Autonomous [[Organizations|Organization]] is a mechanism that enables online communities to form and coordinate economically. -- DAOs make it possible for an online group with members from anywhere in the world to pool capital and hard-code rules — entirely in software — for how that capital will be managed and deployed. Those rules are then enforced by the underlying blockchain. +- DAOs make it possible for an online group with members from anywhere in the world to pool capital and hard-code rules — entirely in software — for how that capital will be managed and deployed. Those rules are then enforced by the underlying [[Blockchain|blockchain]]. ## Resources diff --git a/Decentralized Protocols.md b/Decentralized Protocols.md index de013b2..19b8eb2 100644 --- a/Decentralized Protocols.md +++ b/Decentralized Protocols.md @@ -11,12 +11,12 @@ - [Community](https://x.com/balajis/status/1310101055816921090): open source all code, and issue a token or charge for access to the community. - Open Source projects would have a protocol. You could buy shares of Kubernetes, TensorFlow, ... or contribute to gain tokens. This [[incentives|incentivizes]] contributing and helping people. - [Many more](https://youtu.be/Axj8NJXnCN0)! -- Tokens create new, scoped economies, and those economies enable new ways of organizing production and operation of goods and services. Because these economies are programmable, they can also embed and optimize for value systems and goals. +- Tokens create new, scoped economies, and those economies enable new ways of organizing production and operation of goods and services. Because these economies are programmable, they can also embed and optimize for [[Values|value]] systems and [[Goals|goals]]. - [Moving to protocols, not platforms](https://knightcolumbia.org/content/protocols-not-platforms-a-technological-approach-to-free-speech), is an approach for free speech in the twenty-first century. Rather than relying on a "marketplace of [[ideas]]" within an individual platform — which can be hijacked by those with malicious intent—protocols could lead to a marketplace of ideals, where competition occurs to provide better services that minimize the impact of those with malicious intent, without cutting off their ability to speak entirely. - The fundamental power of the internet is its _interoperability_. It was born out of the ability of different networks to talk to each other using common protocols. The interoperability is what we've lost in the Web 2.0 era. Even such a quintessential thing as a web API has no well-defined standard or protocol, just a very vague concept of REST or RPC. We need commonly accepted standards and _decentralized_ protocols: for web APIs, for identity management, for message queuing, for web callbacks (webhooks), for online transactions, for semantic web and ontology, etc. - Sovereignty through ownership risks being a fantasy for all but the wealthiest. Sovereignty through interoperability may be achievable for nearly everyone. - The question isn't ["Can we own our stack?"](https://www.techpolicy.press/the-path-to-a-sovereign-tech-stack-is-via-a-commodified-tech-stack/) but "Can we move our workloads?" - - The metric for success should focus on making it as simple as possible to move data and applications across suppliers. + - The metric for success should [[Focus|focus]] on making it as simple as possible to move data and applications across suppliers. - [De facto standards almost always beat untested de jure standards](https://www.techpolicy.press/the-path-to-a-sovereign-tech-stack-is-via-a-commodified-tech-stack/). - [Use protocols, not services](https://notnotp.com/notes/use-protocols-not-services/). Centralized services are easy pressure points for compelled [[Identity]] checks and policy mandates. Protocols like IRC, XMPP, ActivityPub, Nostr, and Matrix have no single entity to compel. Pressure must happen server by server across jurisdictions. Choosing protocols over services reduces forced identification risk and improves [[Privacy]] and credible exit. - [Decentralization enables permissionless innovation](https://newsletter.squishy.computer/p/decentralization-enables-permissionless): @@ -39,7 +39,7 @@ - One wallet / DID / private key could allow you to log in to any service. That's your credentials. [[NFTs|Owning a thing]] could allow you to enter somewhere. - Protocols to structure who can do what. Protocol designs are often about the how, but the consequence of the how is an authority model. - There should be no technical or social single-point-of-failure for the overall protocol and network. There should be no single organization or individual who can entirely exclude others from the ecosystem (though the ecosystem may collectively exclude bad actors). There should be multiple independent interoperating service providers for each infrastructure component. -- [Open source protocols should favor composability over just about everything](https://youtu.be/TdBTJY-G8xs). Breaking big things into smaller things. This encourages experimentation at multiple levels. +- [Open source protocols should favor composability over just about everything](https://youtu.be/TdBTJY-G8xs). Breaking big things into smaller things. This encourages [[Experimentation|experimentation]] at multiple levels. - Forking should be a right. Keeps authority contingent (if they abuse power, they might get forked). - **[Progressive decentralization](https://a16z.com/progressive-decentralization-a-playbook-for-building-crypto-applications/) means optional centralization.** - A decentralized protocol can work with a centralized provider. It has the benefits of both (might be fast but no lock-in). @@ -83,8 +83,8 @@ - Shift to a zero-trust model: trust people via signatures, not servers - Build webs of trust via PGP-style key exchanges - Layer reputation & attestations (proof-of-humanity, moderation tags) on keys - - Digital journals as self-sovereign stores of evolving identity data over time. - - Knowledge graphs tracking which peers mirror each datum, enabling witness-based verification. + - Digital [[Journaling|journals]] as self-sovereign stores of evolving identity data over [[Time|time]]. + - [[Knowledge Graphs|Knowledge graphs]] tracking which peers mirror each datum, enabling witness-based verification. - Credit graphs encoding degrees of trust/credit between participants as identity primitives. - [Intersectional Social Data](https://www.radicalxchange.org/media/blog/2019-10-24-uh78r5/) enables group-based and non-individualistic verification via social trust paths. - The ideal is to focus on Keys, not IDs. Don't focus on who you are but what you are authorized to do. @@ -93,7 +93,7 @@ - [Credible exit](https://newsletter.squishy.computer/p/credible-exit) ensures users can leave a service without vendor lock-in by relying on open, decentralized protocols and interchange formats. - Decoupling identifiers from hosting (DNS, CIDs) enables seamless migration between providers by updating name mappings. - Federated protocols (email, Mastodon) support portability of social graphs and messages across services. - - Standard export/import formats (OPML for podcasts, XML for WordPress) facilitate hostile interoperability. + - Standard export/import formats (OPML for [[Podcasts|podcasts]], XML for WordPress) facilitate hostile interoperability. - Durable, universally useful formats (plain text, CSV, PNG, MP3) make data portable and usable outside the originating app. - Local-first storage and immutable data (CRDTs) support ongoing synchronization and avoid stale static exports. - Permissionless, open APIs and protocols allow multiple apps to interoperate without centralized lock-in. diff --git a/Decentralized Web.md b/Decentralized Web.md index d7b96e7..6c03e62 100644 --- a/Decentralized Web.md +++ b/Decentralized Web.md @@ -2,12 +2,12 @@ - The decentralized web is more than [[Blockchain]] and [[Cryptocurrencies]]. [It means building in public, new types of incentives, real ownership of digital property, internet-wide login / identity](https://ckarchive.com/b/p9ueh9hp409k), ... - Key decisions associated with maintaining and updating cryptonetwork protocols. As with every human organization, aligning interests and beliefs among human participants to reach agreement is the goal of [[governance]]. - - [In cryptonetworks, these decisions are made by the community, using open and transparent mechanisms](https://onezero.medium.com/why-decentralization-matters-5e3f79f7638e). DAOs don't need a boss! + - [In cryptonetworks, these decisions are made by the community, using open and transparent mechanisms](https://onezero.medium.com/why-decentralization-matters-5e3f79f7638e). [[Decentralized Autonomous Organizations|DAOs]] don't need a boss! - Cryptocurrencies have the [[Incentives]] to keep the users happy since forking is an option. - [Web3 is a decentralized internet. Under Web3, the internet is shared online and governed by the collective "we"](https://docs.google.com/document/d/1SWJw_NTyUvgdB_asRzsnVyKjciW8dZbeqQeUeWsEiQc). - The decentralized web incentivizes open source. That creates an ecosystem of composable and permissionless tools that increases the speed of innovation. - [Openness allows local groups to design systems for governance, identity, and other needs in ways that are compatible with local goals](https://vitalik.eth.limo/general/2025/09/24/openness_and_verifiability.html). -- A way to look at the decentralized web movement is as a laboratory for economic, political, and social experiments. +- A way to look at the decentralized web movement is as a laboratory for economic, political, and social [[Experimentation|experiments]]. - The space encourages experimentation in a diverse set of fields like [[Governance]] or [[Finances]]. - Networks aren't evenly distributed in the world. E.g: [we see the emergence of densely connected hubs—centralized islands in the internet](https://newsletter.squishy.computer/p/centralization-is-inevitable). Scale-free networks are power-law distributed and emerge due to preferential attachment (rich get richer feedback loops, also known as [[Network Effects]]). @@ -105,7 +105,7 @@ There is an emerging landscape of [Data and Blockchain](https://web.archive.org/ - [Flipside](https://flipsidecrypto.xyz/). - [MetricsDAO](https://metricsdao.notion.site/metricsdao/Bounty-Programs-d4bac7f1908f412f8bf4ed349198e5fe). -#### Data Engineering Folks +#### [[Data Engineering]] Folks - [Yaroslav Tkachenko](https://sap1ens.com/blog/2022/04/11/why-im-joining-goldsky/) - [Mike Ritchie](https://twitter.com/thisritchie/status/1533434879773290496) diff --git a/Deep Funding.md b/Deep Funding.md index b3c9cc5..382037a 100644 --- a/Deep Funding.md +++ b/Deep Funding.md @@ -2,7 +2,7 @@ The goal of [Deep Funding](https://deepfunding.org/) is to develop a system that can allocate resources to public goods (discover public goods value signals) with a level of accuracy, fairness, and open access that rivals how private goods are funded by markets, ensuring that high-quality open-source projects can be sustained. Traditional price signals don't exist, so we need "artificial markets" that can simulate the information aggregation properties of real markets while being resistant to the unique failure modes of public goods funding. [Deep Funding is an Impact Evaluator](https://hackmd.io/@dwddao/HypnqpQKke). -In Deep Funding, multiple mechanisms (involving data, mechanism design, and open source) work together. Each layer can be optimized and iterated independently. +In Deep Funding, multiple mechanisms (involving data, [[Mechanism Design|mechanism design]], and open source) work together. Each layer can be optimized and iterated independently. 1. A mechanism that generates an up-to-date and comprehensive DAG of relevant dependencies given a source node 2. A mechanism that fills the graph with relevant weights. These weights represent the latent item utilities. There can be many ways of getting to them! @@ -13,13 +13,13 @@ In Deep Funding, multiple mechanisms (involving data, mechanism design, and open - Having experts fill weights manually 3. A mechanism that takes that weight vector as input and distributes money to the projects -Deep Funding can be viewed as a [Software-2.0](https://karpathy.medium.com/software-2-0-a64152b37c35) approach to public-goods allocation. Instead of manually designing funding rules, evaluation processes, and governance structures, define an objective function, tests, eval sets, and scoring criteria. Then, let any kind of mechanism (AI models, prediction markets, statistical algorithms, human raters, etc.) compete to solve them. The human work shifts from hand-crafting decision procedures to specifying what "good allocation" looks like and collecting high-quality data. Everything else becomes an optimization problem where participants will try to produce weight predictions that best fit the data. Deep Funding can be seen as **an evolving benchmark suite for truthfully estimating public-goods value**, and progress comes from iterating on the evals rather than hard-coding the system itself. In this view, Deep Funding produces competing value models rather than a single canonical one. +Deep Funding can be viewed as a [Software-2.0](https://karpathy.medium.com/software-2-0-a64152b37c35) approach to public-goods allocation. Instead of manually designing funding rules, evaluation [[Processes|processes]], and [[Governance|governance]] structures, define an objective function, tests, eval sets, and scoring criteria. Then, let any kind of mechanism (AI models, prediction markets, statistical algorithms, human raters, etc.) compete to solve them. The human work shifts from hand-crafting decision procedures to specifying what "good allocation" looks like and collecting high-quality data. Everything else becomes an optimization problem where participants will try to produce weight predictions that best fit the data. Deep Funding can be seen as **an evolving benchmark suite for truthfully estimating public-goods value**, and progress comes from iterating on the evals rather than hard-coding the system itself. In this view, Deep Funding produces competing value models rather than a single canonical one. ## Desired Properties - Credible Neutrality Through Transparent and Simple Mechanisms -- Comparative Truth-Seeking Over Absolute Metrics -- Plurality-Aware Preference Aggregation +- Comparative Truth-Seeking Over Absolute [[Metrics]] +- [[Plurality]]-Aware Preference Aggregation - Collusion-Resistant Architecture - Practical Scalability - Credible Exit and Forkability @@ -48,7 +48,7 @@ After participating in the ML competition and Prediction Market, and doing a few - **Juror Reliability** - So far, expert jurors' pairwise comparisons have been inconsistent, noisy, and low in statistical power - - Getting comparisons has been quite expensive in time and resources + - Getting comparisons has been quite expensive in [[Time|time]] and resources - The jury (secret) pool diversity is not guaranteed - Asking jurors "how much better" introduces order‑dependence and scale mismatch - Messy jurors have [disproportionate impact on the weights](https://davidgasquez.github.io/deepfunding-trial-data-analysis/#-robustness-checks) @@ -109,7 +109,7 @@ After participating in the ML competition and Prediction Market, and doing a few - Rather than a single weighting mechanism, allow different communities to define their own evaluation functions (security-focused, innovation-focused, stability-focused) that can be composed. This enables plurality while maintaining comparability - Multiple weight models compete simultaneously and funding allocators choose among them. - Create a bounty system where anyone can claim rewards for discovering hidden dependencies (similar to bug bounties) - - This crowdsources the graph discovery problem and incentivizes thorough documentation. + - This crowdsources the graph discovery problem and incentivizes thorough [[Documentation|documentation]]. - Projects can opt out of the default distribution and declare a custom one for dependencies. Organizers can allow or ignore that - Self-declaration needs a "contest process" to resolve issues/abuse - Harberger Tax on self-declarations? Bayesian Truth Serum for Weight Elicitation? diff --git a/Design Docs.md b/Design Docs.md index 299d756..d86ef9b 100644 --- a/Design Docs.md +++ b/Design Docs.md @@ -2,12 +2,12 @@ [Define software designs through design docs](https://www.industrialempathy.com/posts/design-docs-at-google/). -- These are relatively informal documents that the primary author or authors of a software system or application create before they embark on the coding project. If code is the bricks and mortar, then these docs are the blueprints. +- These are relatively informal documents that the primary author or authors of a software [[Systems|system]] or application create before they embark on the coding project. If code is the bricks and mortar, then these docs are the blueprints. - Short specs are more likely to be read. The purpose of a spec is to briefly communicate the "why", "what" and "how" of the project to the rest of the team. Ideally these short documents force teams to scope out work so priorities are clear and teams avoid building the wrong thing. - [The longer your document is, the more likely people will only read the comments and not the body.](https://twitter.com/hamiltonulmer/status/1562817324184440832) - A [[Technical Specifications|technical specification]] forces you to think through complicated issues and to get everyone on the same page. This helps to avoid wasting time on dead-end solutions or building the wrong thing. - It's hard to make technical decisions while remote. [Build a Proposal Culture](https://web.archive.org/web/20220925011031/https://www.hamiltonulmer.com/writing/building-a-proposal-culture) to enable effective distributed technical decision making via [[Writing]] and collecting [[Feedback]] on a written document in an inclusive, async-friendly way. -- Even if no one else reads them, they force you to clarify your thinking before you start the (more expensive) process of implementation. +- Even if no one else reads them, they force you to clarify your [[Thinking|thinking]] before you start the (more expensive) process of implementation. - Design docs fulfill the following functions in the software development life-cycle: - Early identification of design issues when making changes is still cheap. - Achieving consensus around a design in the organization. @@ -17,12 +17,12 @@ - Write them in whatever form makes the most sense for the particular project. - A good-to-start-with structure can be: 1. Context and scope. Overview of the landscape in which the new system is being built and what is actually being built. Focused on objective background facts. Keep it short. - 2. Goals and non-goals. What the goals of the system are, and, sometimes more importantly, what non-goals are. + 2. [[Goals]] and non-goals. What the goals of the system are, and, sometimes more importantly, what non-goals are. 3. Design. This is the place to write down the trade-offs you made in designing your software. Given the context (facts), goals and non-goals (requirements), the design doc is the place to suggest solutions and show why a particular solution best satisfies those goals. 4. APIs. If the system under design exposes an API, then sketching out that API is usually a good idea. 5. Data storage. Systems that store data should likely discuss how and in what rough form this happens. 6. Alternatives considered. Share alternative designs that would have reasonably achieved similar outcomes. - 7. Cross-cutting concerns. This is where your organization can ensure that certain cross-cutting concerns such as security, privacy, and observability are always taken into consideration. These are often relatively short sections that explain how the design impacts the concern and how the concern is addressed. Teams should standardize what these concerns are in their case. + 7. Cross-cutting concerns. This is where your organization can ensure that certain cross-cutting concerns such as security, [[Privacy|privacy]], and observability are always taken into consideration. These are often relatively short sections that explain how the design impacts the concern and how the concern is addressed. Teams should standardize what these concerns are in their case. - In many docs a diagram can be useful. - The steps in the life-cycle of a design document are: Create, Iterate, Review, Implement, Iterate and Learn. - [The RFC and feedback should be posted publicly. Everyone can join the discussion. The goal is to include as many people as possible to access more points of view and spread the knowledge simultaneously](https://candost.blog/how-to-stop-endless-discussions/). diff --git a/Documentation.md b/Documentation.md index 8b399c2..eb194c2 100644 --- a/Documentation.md +++ b/Documentation.md @@ -1,14 +1,14 @@ # Documentation -- If your product/tool/process documentation is not good enough, people will not use it. -- The documentation for a project should live in the same repository as the code itself. +- If your product/tool/[[Processes|process]] documentation is not good enough, people will not use it. +- The documentation for a project should live in the same repository as the [[Programming|code]] itself. - Update documentation in lockstep when updating the codebase. - Documentation should be **versioned**. - Documentation changes should be reviewed in the same way as your code. - Show realistic examples and verify them against the code when possible with type checks, tests, or builds. -- [Documentation](https://diataxis.fr/) needs to include and be structured around its four different functions: [tutorials, how-to guides, technical reference and explanation](https://documentation.divio.com/introduction/). Each of them requires a distinct mode of writing. - - A tutorial is learning-oriented, allows the newcomer to get started, similar to a lesson. E.g: teaching a small child how to cook. - - A how-to guide is goal-oriented, shows how to solve a specific problem with a series of steps. E.g: a recipe in a cookery book. +- [Documentation](https://diataxis.fr/) needs to include and be structured around its four different functions: [tutorials, how-to guides, technical reference and explanation](https://documentation.divio.com/introduction/). Each of them requires a distinct mode of [[Writing|writing]]. + - A tutorial is [[Learning|learning]]-oriented, allows the newcomer to get started, similar to a lesson. E.g: teaching a small child how to cook. + - A how-to guide is [[Goals|goal]]-oriented, shows how to solve a specific problem with a series of steps. E.g: a [[Recipes|recipe]] in a cookery book. - An explanation is understanding-oriented, provides background and context. E.g: an article on culinary social history. - A reference guide is information-oriented, describes the machinery and is accurate and complete. E.g: a reference encyclopedia article. - For reference docs, use a [predictable structure](https://raw.githubusercontent.com/tempoxyz/mpp/refs/heads/main/AGENTS.md): one-line summary, usage, return type or output, parameters or inputs, and clearly marked optional fields. @@ -28,7 +28,7 @@ - Don't claim things are "easy" or "fast." - Avoid future tense ("will"), weak conditional phrasing ("should"), and Latin abbreviations like "e.g."; prefer direct instructions and "for example". - Keep headings short and never skip heading levels. -- Programming is fundamentally about building a theory. A shared mental model of how a system works, why it works that way, and how it should evolve. +- Programming is fundamentally about building a theory. A shared [[Mental Models|mental model]] of how a [[Systems|system]] works, why it works that way, and how it should evolve. - The source code is merely a written representation of this theory, and like all representations, it's lossy. - Critical knowledge about intent, design decisions, trade-offs, and the reasoning behind architectural choices exists only in the minds of the people who built the system. - [As you are working in a team, then you have to address the problem of shared understanding](https://surfingcomplexity.blog/2022/11/24/writing-docs-well-why-should-a-software-engineer-care/). This is where documentation comes in. @@ -45,7 +45,7 @@ - Remove weasel words. Replace adjectives with data or details. - Ask the "So what" question to every sentence that you write. - Prefer active voice to passive. -- Most docs are short-lived and used for point-in-time discussions. However, some documents serve for a longer time. To have your writing stand the test of time, avoid using references that change with time or location. +- Most docs are short-lived and used for point-in-[[Time|time]] discussions. However, some documents serve for a longer time. To have your writing stand the test of time, avoid using references that change with time or location. - Documentation is valuable if it captures the design decision and intention at the time of the creation of the software, rather than the functionality of the software itself. ## Resources diff --git a/Double Crux.md b/Double Crux.md index b9854a6..4f53325 100644 --- a/Double Crux.md +++ b/Double Crux.md @@ -2,7 +2,7 @@ [Double crux](https://www.lesswrong.com/posts/exa5kmvopeRyfJgCy/double-crux-a-strategy-for-resolving-disagreement) is a great framework where both parties abstract their arguments by one level and find a falsifiable fact that, if proven true, would cause them to change their beliefs. -- Focus on narrowing the scope of the [[Resolving Disagreement|discussion]]. Find common ground. +- [[Focus]] on narrowing the scope of the [[Resolving Disagreement|discussion]]. Find common ground. - Define terms to avoid getting lost in semantic confusions that miss the real point. - Ask "What would need to be true for you to change your mind?". - Find specific test cases. diff --git a/Dungeons and Dragons.md b/Dungeons and Dragons.md index 2d8f664..c0b9600 100644 --- a/Dungeons and Dragons.md +++ b/Dungeons and Dragons.md @@ -1,6 +1,6 @@ # Dungeons and Dragons -Dungeons & Dragons is a cooperative storytelling game that harnesses your imagination and invites you to explore a fantastic world of adventure, where heroes battle monsters, find treasures, and overcome epic quests. +Dungeons & Dragons is a cooperative [[Social Games|storytelling game]] that harnesses your imagination and invites you to explore a fantastic world of adventure, where heroes battle monsters, find treasures, and overcome epic quests. - [Dungeons & Dragons 5th Edition wiki](http://dnd5e.wikidot.com/). - [Dungeons & Dragons Beyond Essentials](https://www.dndbeyond.com/essentials). diff --git a/Emergence.md b/Emergence.md index e98406a..e6082a2 100644 --- a/Emergence.md +++ b/Emergence.md @@ -9,8 +9,8 @@ When things interact, they often birth new, unpredictable forms. **The sum total You can't top-down plan emergent [[systems]], but you can provoke them into being. [Here are some attributes that emergent systems often have](http://gordonbrander.com/pattern/provoking-emergence/): - [[Evolution]] -- More than one goal that can be achieved in more than one way. +- More than one [[Goals|goal]] that can be achieved in more than one way. - Alphabet: An alphabet is a kit of parts, together with rules for combining them (e.g Lego). The simpler the building blocks, the better. - A holon is something that is simultaneously a whole in and of itself, as well as a part of a larger whole (e.g: humans). -- An easy initial learning curve but infinite potential (e.g. Minecraft). +- An easy initial [[Learning|learning]] curve but infinite potential (e.g. Minecraft). - Involves multiple agents. It has [[network effects]]. diff --git a/Engineering Process.md b/Engineering Process.md index ce525f4..0f45cb1 100644 --- a/Engineering Process.md +++ b/Engineering Process.md @@ -1,6 +1,6 @@ # Engineering Process -1. **Issue driven development**. Always create an issue and assign it to you for anything you work on. If it is worth spending time on, it is worth creating an issue for it since that enables other people to learn and help. You can always edit the description or close it when the problem is something different or disappears. +1. **Issue driven development**. Always create an issue and assign it to you for anything you work on. If it is worth spending [[Time|time]] on, it is worth creating an issue for it since that enables other people to learn and help. You can always edit the description or close it when the problem is something different or disappears. 2. When working on an issue, make sure that it contains the latest information. The issue should be the source of truth. 3. After a discussion related to an issue, update the issue body with the consensus or final conclusions. This makes it much easier to see the current state of an issue for everyone involved in the implementation and prevents confusion later on. 4. Submit the smallest item of work that makes sense. When creating an issue describe the smallest fix possible, put suggestions for enhancements in separate issues and link them. diff --git a/Evolution.md b/Evolution.md index 1489408..fbaa5fb 100644 --- a/Evolution.md +++ b/Evolution.md @@ -2,7 +2,7 @@ > If you want to make a living flower, you don't build it - you grow it from the seed. Christopher Alexander. -- Evolution is a behavior that emerges in any system that has: +- Evolution is a behavior that [[Emergence|emerges]] in any system that has: 1. Mutation 2. Heredity 3. Selection diff --git a/Experimentation.md b/Experimentation.md index 764e378..5442b13 100644 --- a/Experimentation.md +++ b/Experimentation.md @@ -12,7 +12,7 @@ When you're not certain of the right answer, the best approach is to [have a por Normally this will consist of two groups: A **control** and **test** group, but it could also be implemented with multiple test groups (a multivariate test). The goal of an A/B test is to reach a statistically significant result, i.e. you can say that one variant is better than the other with a high level of confidence that observed difference did not occur by chance. While A/B testing is a very powerful conversion optimization instrument, it requires lots of hard work. [Most of your experiments will not produce a significant result](https://www.jitbit.com/news/185-most-of-your-ab-tests-will-fail). -Tooling built on the right principles can provide the foundation for a culture of experimentation. Experimentation platforms should be transparent and open such that anyone can create an experiment and everyone can see the historical experiments that have been run — including hypothesis, experimental design, and results. Social layers such as commenting and version control allow for peer review and peer-to-peer education. Sharing functionality to make it easy to link to experiments in emails, Slack, or other communication channels reinforces experimental literacy. Workflows that abstract statistical concepts and make it easier for people to understand what experiments are reduce the perceived barrier to entry and make it easier for people to engage. [Altogether, these principles help cultivate and reinforce experimental literacy within your organization](https://towardsdatascience.com/the-experimentation-gap-3f5d374d354c). +Tooling built on the right principles can provide the foundation for a [[Culture|culture]] of experimentation. Experimentation platforms should be transparent and open such that anyone can create an experiment and everyone can see the historical experiments that have been run — including hypothesis, experimental design, and results. Social layers such as commenting and version control allow for peer review and peer-to-peer education. Sharing functionality to make it easy to link to experiments in emails, [[Slack]], or other [[Communication|communication]] channels reinforces experimental literacy. Workflows that abstract statistical concepts and make it easier for people to understand what experiments are reduce the perceived barrier to entry and make it easier for people to engage. [Altogether, these principles help cultivate and reinforce experimental literacy within your organization](https://towardsdatascience.com/the-experimentation-gap-3f5d374d354c). ## Running Experiments @@ -20,14 +20,14 @@ We do not live in an ideal world, so we need to be very deliberate and thoughtfu - Generate hypotheses based on actual observations. All experiments should have a hypothesis in plain English. - Do not simply copy what other companies are doing (do not assume that other companies tested and validated something). -- Choose one primary metric in advance to determine "winner" while keeping a few guardrail metrics. +- Choose one primary [[Metrics|metric]] in advance to determine "winner" while keeping a few guardrail metrics. - [You're probably **not measuring what you thought you were measuring**](https://www.lesswrong.com/posts/9kNxhKWvixtKW5anS/you-are-not-measuring-what-you-think-you-are-measuring). But if you measure enough different stuff, you might figure out what you're actually measuring. - Log generously. -- Stack rank all ideas based on level of effort and potential impact -- also required sample size/run time to get results. +- Stack rank all [[Ideas|ideas]] based on level of effort and potential impact -- also required sample size/run [[Time|time]] to get results. - Continuously validate tracking implementation. - Think carefully when choosing your randomization unit. - Run more A/A tests. -- The best two places to run an A/B test: In early planning (what kinds of stories do people want) and when adding finishing touches (alternative versions of a scene). +- The best two places to run an A/B test: In early [[Planning|planning]] (what kinds of stories do people want) and when adding finishing touches (alternative versions of a scene). - Remember, you're measuring averages. - A great way to help people understand the importance of healthy scrutiny and [how our biases work](https://www.goodreads.com/book/show/11468377-thinking-fast-and-slow) can be to do a session where people guess the impact of an A/B test before you reveal it. diff --git a/Federated Networks.md b/Federated Networks.md index 2baa783..5212acc 100644 --- a/Federated Networks.md +++ b/Federated Networks.md @@ -4,10 +4,10 @@ - This gives users more choices for applications, policies, and community cultures. Email is an example of a federated protocol that everyone on the internet uses. Gmail is a popular email application, but if you use a different provider you can still communicate with anyone with an email address. - Federated networks provide a familiar user experience, since users do not have to bear full responsibility for their account credentials, and can interact with content the way they're used to. - On the downside, a federated protocol might be slow to adapt while a centralized service is able to iterate into the modern world and beyond. WhatsApp was able to introduce end-to-end encryption to over a billion users with a single software update (email is still not end-to-end encrypted). Federated services always seem to coalesce around a provider (or instance) that the bulk of people use, federation becomes a sort of implicit threat. [An open source infrastructure for a centralized network now provides almost the same level of control as federated protocols, without giving up the ability to adapt. If a centralized provider with an open source infrastructure ever makes horrible changes, those that disagree have the software they need to run their own alternative instead. It may not be as beautiful as federation, but at this point it seems that it will have to do.](https://signal.org/blog/the-ecosystem-is-moving/) -- It also has [a lot of potential](https://drewdevault.com/2020/09/20/The-potential-of-federation.html). Because there are hundreds or even thousands of instances, the users get the privilege of choosing an instance whose rules they like, and which federates with other instances they wish to talk to. This system also makes it hard for marketing and spam to get a foothold — it optimizes for a self-governing system of human beings talking to human beings, and not for corporations to push their products. +- It also has [a lot of potential](https://drewdevault.com/2020/09/20/The-potential-of-federation.html). Because there are hundreds or even thousands of instances, the users get the privilege of choosing an instance whose rules they like, and which federates with other instances they wish to talk to. This system also makes it hard for marketing and spam to get a foothold — it optimizes for a self-governing system of human beings [[Talking|talking]] to human beings, and not for corporations to push their products. - The [[governance]] of a federated system then becomes distributed among many operators. Every instance has the following privileges: - To set the rules which govern users of their instance. - To set the rules which govern who they federate with. - It encourages fractal [[governance]] (e.g: communist at family level, liberal at state level, ...) - - Users can choose servers based on trust, privacy policies, and community standards, which can enhance security and user control over data. + - Users can choose servers based on trust, [[Privacy|privacy]] policies, and community standards, which can enhance security and user control over data. - [The network needs a way to preserve content when a node goes down (probably using content-addressed storage)](https://dustycloud.org/blog/how-decentralized-is-bluesky/). diff --git a/Feedback Loops.md b/Feedback Loops.md index d67cb8d..0eb247a 100644 --- a/Feedback Loops.md +++ b/Feedback Loops.md @@ -7,9 +7,9 @@ - **[[Network Effects|Things are connected]]**. Changing one variable in a [[Systems|system]] will affect other variables in that system and other systems. This is important because it means that designers must not only consider particular elements of a design, but also their relation to the design as a whole and to the greater environment. - All complex [[systems]] are subject to positive and negative feedback loops whereby A causes B, which in turn influences A (and C), and so on – with higher-order effects frequently resulting from continual movement of the loop. - Feedback loops vary in their accuracy. - - Accurate feedback means that it **reliably** and **clearly** tells you when you do something right. If you get the quadratic formula wrong, you can check the right formula and know what was wrong. + - Accurate [[Feedback|feedback]] means that it **reliably** and **clearly** tells you when you do something right. If you get the quadratic formula wrong, you can check the right formula and know what was wrong. - An inaccurate feedback loop means that the results of the evaluation phase are “**noisy**” and contain significant variance, so the next cycle will need to take that into account. E.g: playing bowls without a coach. - - Learning under conditions of noisy data starts with world construction. Imagine a possible future, and repeat this to generate hundreds of possible future worlds. The main skills and resources required are creativity, [[slack]], and [equanimity](https://en.wikipedia.org/wiki/Equanimity). Creativity leads to a higher rate of idea generation and [[slack]] gives us more time to generate ideas. Equanimity is important because it allows us to persevere in the absence of tangible feedback. + - [[Learning]] under conditions of noisy data starts with world construction. Imagine a possible [[Future|future]], and repeat this to generate hundreds of possible future worlds. The main skills and resources required are creativity, [[slack]], and [equanimity](https://en.wikipedia.org/wiki/Equanimity). Creativity leads to a higher rate of idea generation and [[slack]] gives us more [[Time|time]] to generate [[Ideas|ideas]]. Equanimity is important because it allows us to persevere in the absence of tangible feedback. - [The shorter and more accurate the feedback loop, the easier it is to learn](http://web.archive.org/web/20240425163013/https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loops/). The **tighter** your feedback loop, the better your work. - Fast and accurate loops might get you on a local maximum or might not work when the underlying system is noisy. @@ -17,5 +17,5 @@ - Poor kids becoming rich adults. Then, they raise rich kids that don't share the same [[Values]] that made the parents rich. - Artist making music while they experience life without fame. Once fame gets in they don't have the same life experience and can't produce the same type of music. -- [The Mongol Empire was an unification of many nomadic tribes. They were unstoppable for a while until they started to become more civilized. The thing that made them unstoppable (nomadic and "uncivilized") stopped being a part of their [[Culture]] and the empire began to break down.](https://www.dancarlin.com/product/hardcore-history-wrath-of-the-khans-series/) -- The [[Science#Metascience]] learning loop: trial new social processes → evaluate effectiveness → scale successful approaches → amplify transformative changes. A positive feedback loop designed to systematically improve how science itself operates. +- [The Mongol Empire was an unification of many nomadic tribes. They were unstoppable for a while until they started to become more civilized. The thing that made them unstoppable (nomadic and "uncivilized") stopped being a part of their culture and the empire began to break down.](https://www.dancarlin.com/product/hardcore-history-wrath-of-the-khans-series/) +- The [[Science#Metascience]] learning loop: trial new social [[Processes|processes]] → evaluate effectiveness → scale successful approaches → amplify transformative changes. A positive feedback loop designed to systematically improve how science itself operates. diff --git a/Feedback.md b/Feedback.md index 9a01f1f..af876ee 100644 --- a/Feedback.md +++ b/Feedback.md @@ -1,6 +1,6 @@ # Feedback -Feedback is the core of personal and professional growth. Feedback helps us be better at what we do by establishing clear communications and expectations. +Feedback is the core of personal and professional growth. Feedback helps us be better at what we do by establishing clear [[Communication|communications]] and expectations. All feedback is good feedback. If you agree with it, it tells you something about yourself which you can work on correcting. If you don't agree with it, it tells you that there is an incorrect perception about you in the other person's mind. You can also work on correcting that. @@ -8,32 +8,32 @@ All feedback is good feedback. If you agree with it, it tells you something abou - When someone tells you how to fix it, they're almost always wrong. - Collect feedback from everybody. -Your goal is not to do what _you_ think is best, it's to help _others_. This includes respecting their preferences, and respecting their autonomy. It's key that you listen to feedback, be open to the possibility that your actions are systematically unhelpful, and work to build better models of your friends and their preferences. In an ideal world I'd only take the actions that are net good, and avoid all of the ones that are net bad, but in a limited information world this is impossible. And empirically, actually trying _far_ outweighs not trying at all. But you still want to get as net good as possible! +Your goal is not to do what _you_ think is best, it's to help _others_. This includes respecting their preferences, and respecting their autonomy. It's key that you [[Listening|listen]] to feedback, be [[Openness|open]] to the possibility that your actions are systematically unhelpful, and work to build better [[Mental Models|models]] of your friends and their preferences. In an ideal world I'd only take the actions that are net good, and avoid all of the ones that are net bad, but in a limited information world this is impossible. And empirically, actually trying _far_ outweighs not trying at all. But you still want to get as net good as possible! ## Taking Feedback - Use the word advice instead of feedback to get more results. - The only constant in the world is change. Be open to it and accept it with a smile. - Understand and accept that you will make mistakes. -- Be mindful. Develop meta-awareness around the areas that you received feedback. +- Be [[Mindfulness|mindful]]. Develop meta-awareness around the areas that you received feedback. - Look for opportunities to _stop doing_ or _start doing_ critiqued behaviors. ### Technical Feedback -- You are not your work (code, design, ...). +- You are not your work ([[Programming|code]], design, ...). - No matter how much you know about something, someone else will always know more. - Treat people who know less than you with respect, deference, and patience. - True authority stems from knowledge, not from position. - Critique code instead of people — be kind to the coder, not to the code. -- Focus on [Empathy](http://bravenewgeek.com/engineering-empathy/). +- [[Focus]] on [Empathy](http://bravenewgeek.com/engineering-empathy/). ## Giving Feedback - Feedback needs to be informal, frequent, and done authentically. - Feedback should be offered with a willingness to listen in return. Those giving feedback should also ask for it. -- Challenge ideas, not people. Address behavior, but don't label people. +- Challenge [[Ideas|ideas]], not people. Address behavior, but don't label people. - Most feedback you deliver should be positive. This makes the negative feedback more important. -- Prepare. What do you value in someone? Where do you think their biggest opportunities to improve are? +- Prepare. What do you [[Values|value]] in someone? Where do you think their biggest opportunities to improve are? - **Keep the feedback actionable, specific, and kind.** Be as specific as possible and contain concrete suggestions on how to improve. - Imagine what things feel like from the other perspective. - Criticize in private, praise in public. diff --git a/Finances.md b/Finances.md index 1c1e644..7ae8383 100644 --- a/Finances.md +++ b/Finances.md @@ -5,7 +5,7 @@ - [Be boring with most of your portfolio](https://twitter.com/VitalikButerin/status/1743991285214527532). - Spend (relatively) more money on things you interact with the most, time-wise (as well as intensity-wise). - Up to a point, being richer will not necessarily make you happier. -- Spend on others, especially people you are close to. Positive feedback loop: Prosocial spending makes you happier, and happiness makes you more likely to spend pro-socially. +- Spend on others, especially people you are close to. Positive [[Feedback Loops|feedback loop]]: Prosocial spending makes you happier, and happiness makes you more likely to spend pro-socially. ## Resources diff --git a/Fitness.md b/Fitness.md index a220894..f78eda7 100644 --- a/Fitness.md +++ b/Fitness.md @@ -18,7 +18,7 @@ - Pick certain moments where you would always check your posture. Walking through doorways, standing up, ... - Another great tip is to brush your teeth with [your back against the wall](https://www.youtube.com/watch?v=GbGSvAEkE68) for 3 minutes. - Exercise and live a less sedentary lifestyle to improve overall posture. -- Focus on [building core strength](https://www.youtube.com/watch?v=4BOTvaRaDjI). You can [strengthen your back in 5 minutes](https://www.reddit.com/r/bodyweightfitness/comments/fxrooc/how_to_strengthen_your_back_in_5_minutes_a_day/). +- [[Focus]] on [building core strength](https://www.youtube.com/watch?v=4BOTvaRaDjI). You can [strengthen your back in 5 minutes](https://www.reddit.com/r/bodyweightfitness/comments/fxrooc/how_to_strengthen_your_back_in_5_minutes_a_day/). ## Resources diff --git a/Focus.md b/Focus.md index ba63816..06b6c51 100644 --- a/Focus.md +++ b/Focus.md @@ -2,7 +2,7 @@ [Focus (absorption, concentration) is the ability to narrow the attention so as to apply it in a more detailed and penetrating way for sustained periods of time on some chosen part of your present experience](https://www.lesswrong.com/s/xqgwpmwDYsn8osoje/p/35eEHAXis3jMqETod). -- Whatever your primary motivations are in life, you won't get anywhere by waiting for something to happen. Plan! It'll never be the right time. +- Whatever your primary motivations are in life, you won't get anywhere by waiting for something to happen. [[Planning|Plan]]! It'll never be the right [[Time|time]]. - Humans do not think [[Thinking|strategically]] by default. - Environmental changes can make it easier to attend effectively to the right things. - Removing clutter and other distractions can make attention less difficult, for which the virtues of orderliness and simplicity can help. diff --git a/Future.md b/Future.md index d3cad93..d027297 100644 --- a/Future.md +++ b/Future.md @@ -9,14 +9,14 @@ History teaches us that in 100 years from now [[Openness|some of the assumptions - Give birth without advanced assistance. - Not caring for all the [animal suffering in the wild](https://longtermrisk.org/the-importance-of-wild-animal-suffering/). - Nature is not safe! The default is suffering. The current mentality is that nature is good and disruptions from nature are bad. -- The ignorance of Social Media and its [full impact on society](https://twitter.com/M_B_Petersen/status/1483457679800651787). +- The ignorance of [[Social Media Issues|Social Media]] and its [full impact on society](https://twitter.com/M_B_Petersen/status/1483457679800651787). - Is "being bad for society" an emergent property of social networks as they grow? - Current Voting Systems. - Not relying more on tools like Prediction Markets (e.g: [to spot papers that might not replicate](https://vitalik.eth.limo/general/2024/11/09/infofinance.html)). ## Predictions -- More experimentation around [[Politics|governance]]: +- More [[Experimentation|experimentation]] around [[Politics|governance]]: - Charter cities. - [Holacracy](https://en.m.wikipedia.org/wiki/Holacracy). - [[Decentralized Autonomous Organizations|Cooperatives/DAOs]]. @@ -25,11 +25,11 @@ History teaches us that in 100 years from now [[Openness|some of the assumptions - More people aware of the tricks our minds play on us ([Mind Field](https://www.youtube.com/playlist?list=PLZRRxQcaEjA4qyEuYfAMCazlL0vQDkIj2) and [The Story Of Us](https://web.archive.org/web/20230129233043/https://waitbutwhy.com/2019/08/story-of-us.html)) as well as [how diverse humans are](https://youtu.be/fC5qucSk18w). - More people are aware of [how cool win-win games are](https://nonzerosum.games/)! - More interactive explanations like the ones the awesome [Nicky Case](https://ncase.me/) does! -- More concern around systems with weird incentives causing large amounts of pain (Moloch). +- More concern around [[Systems|systems]] with weird [[Incentives|incentives]] causing large amounts of pain (Moloch). - E.g: Criminal system rework. Focusing on the system and not on individual punishment. - Work valuation changes (plumbing more expensive than some software development) due to [Moravec's paradox](https://en.wikipedia.org/wiki/Moravec%27s_paradox). We will automate making a full app before a robot is able to master physical arms and legs like a 5-year-old. -- Open data will be more important as it can produce better models and help coordinate people providing shared context. -- The current decentralized protocols (IPFS, ActivityPub, ...) need to evolve more. Especially around UX. People don't care about decentralization, they care about UX. [Centralization is inevitable](https://newsletter.squishy.computer/p/centralization-is-inevitable). +- [[Open Data]] will be more important as it can produce better models and help coordinate people providing shared context. +- The current [[Decentralized Protocols|decentralized protocols]] ([[IPFS]], ActivityPub, ...) need to evolve more. Especially around UX. People don't care about decentralization, they care about UX. [Centralization is inevitable](https://newsletter.squishy.computer/p/centralization-is-inevitable). ### Exciting Software Engineering Ideas diff --git a/Game Design.md b/Game Design.md index f459fee..d69f39a 100644 --- a/Game Design.md +++ b/Game Design.md @@ -17,6 +17,6 @@ Some [principles](https://youtu.be/zaxAdRlyZQ8) to keep in mind: - **Agency without overwhelm**: Provide meaningful choices but not so many that decision paralysis occurs. Options should feel empowering, not burdensome. - **Joy in discovery**: Don't give people answers they haven't figured out themselves. The process of discovery is often more rewarding than the solution. - **Challenge authenticity**: Bypasses and cheats make challenges feel less real, even if unused. Maintain the integrity of difficulty. -- **Environmental over mental changes**: When designing systems, prefer changing the environment to changing how people think. Less invasive, more ethical. -- **Interpersonal complexity**: People making things difficult for each other prevents boredom. Social dynamics create endless variety. +- **Environmental over mental changes**: When designing [[Systems|systems]], prefer changing the environment to changing how people think. Less invasive, more ethical. +- **Interpersonal complexity**: People making things difficult for each other prevents boredom. [[Social Games|Social dynamics]] create endless variety. - **Sensual bandwidth**: High-bandwidth sensory experiences (tactile, visual, auditory) are essential for engagement and meaning. diff --git a/Gardening.md b/Gardening.md index 641b7f2..47ad648 100644 --- a/Gardening.md +++ b/Gardening.md @@ -7,7 +7,7 @@ - Observe and interact. - Catch and store energy. For example, capturing rain water. - Obtain a yield. - - Apply self-regulation and accept feedback. + - Apply self-regulation and accept [[Feedback|feedback]]. - Use and value renewable resources and services. - Produce no waste. - Design from patterns to details. diff --git a/Git.md b/Git.md index 6527866..7442b6c 100644 --- a/Git.md +++ b/Git.md @@ -22,7 +22,7 @@ 1. Include the purpose of the Pull Request. 2. Provide an overview of why the work is taking place. Give a good background on why the change is happening. -3. Be explicit about what feedback you want, if any: a quick pair of :eyes: on the code, discussion on the technical approach, critique on design, a review of copy. +3. Be explicit about what [[Feedback|feedback]] you want, if any: a quick pair of :eyes: on the code, discussion on the technical approach, critique on design, a review of copy. - [Explicitly list how to perform validation](https://www.brautaset.org/posts/levelling-up-pull-requests.html). 4. [Offering feedback](https://mtlynch.io/human-code-reviews-1/) 5. Familiarize yourself with the context of the issue, and reasons why this Pull Request exists. diff --git a/Goals.md b/Goals.md index 76df749..9135cd1 100644 --- a/Goals.md +++ b/Goals.md @@ -4,7 +4,7 @@ Goals are good for setting a direction. [[Systems]] are for making progress. You - Minimize decision points. The main thing that consumes willpower isn't the process of doing a task, it's deciding to do the task in the first place. The goal is to get stuff done, not to decide to get stuff done. - Shape the default. The ideal situation is for doing the right action to feel like the default. -- You don't hit a quantitative goal by focusing on the goal. You hit a quantitative goal by focusing on the process. +- You don't hit a quantitative goal by focusing on the goal. You hit a quantitative goal by focusing on the [[Processes|process]]. - Beware trivial inconveniences. Think of decisions in terms of activation energy. Some decisions need more energy than others depending on how they are set up. Make the good decision effortless. Remove friction from the path to your goals. - [A better wording for goals is "quests"](https://www.raptitude.com/2024/08/do-quests-not-goals/). A quest is an adventure, and you expect it to be one that changes you (gives a reward), not just your situation. diff --git a/Governance.md b/Governance.md index a024281..5272470 100644 --- a/Governance.md +++ b/Governance.md @@ -3,7 +3,7 @@ Governance is a tool an [[Organizations|organization]] uses to decide where power should reside and who gets to click on what button. This process places rails around operations, which helps an organization get from point A to point B more smoothly. - There is an idea that governance — public or corporate — [is driven by the self-interested effort of leaders to acquire and keep their power](https://fs.blog/the-dictators-handbook/). -- [Governance means conflict](https://twitter.com/armaniferrante/status/1450878886535839745). It also means resolution. Accept the former, focus on the latter, via transparency. +- [Governance means conflict](https://twitter.com/armaniferrante/status/1450878886535839745). It also means resolution. Accept the former, [[Focus|focus]] on the latter, via transparency. - [There is still much room to try different things around Governance](https://www.cold-takes.com/ideal-governance-for-companies-countries-and-more/). - A multi-level liquid democracy where every voter chooses a delegate, and 50-200 delegates can empower second-level delegates, [biases for sophistication](https://vitalik.eth.limo/general/2025/12/17/societies.html) (because each level ends up more sophisticated than the previous) and guards against populism (because a delegate can't amass extreme power by gaining a huge following directly) while avoiding empowering pre-selected aristocracies. diff --git a/Habits.md b/Habits.md index 176e8ab..2d32def 100644 --- a/Habits.md +++ b/Habits.md @@ -4,7 +4,7 @@ Habits can be great because they help us get tasks done efficiently without having to spend willpower (a limited resource) on them all the time. The purpose of a habit is to remove that action from self-negotiation. You no longer expend energy deciding whether to do it ([ego depletion](https://en.wikipedia.org/wiki/Ego_depletion)). [Having a habit collapses hundreds of future decisions into one, and gives you focus](https://twitter.com/kepano/status/1698388043278012621). Habits will keep the frontal cortex free to solve other problems. Make a deliberate choice about what needs consistency and what doesn't. -[Make it part of your identity to be an agent](https://www.neelnanda.io/blog/become-a-person-who-actually-does-things) that does things. E.g: noticing the small problems, and fixing them. The most effective way to change your habits is to focus not on what you want to achieve, but on who you wish to become. Focus on [[The Four Laws of Behavior Change]]. [Make a habit of trying safe, low-stakes experiments, to learn more about yourself & what works for you](https://blog.ncase.me/30). +[Make it part of your identity to be an agent](https://www.neelnanda.io/blog/become-a-person-who-actually-does-things) that does things. E.g: noticing the small problems, and fixing them. The most effective way to change your habits is to [[Focus|focus]] not on what you want to achieve, but on who you wish to become. Focus on [[The Four Laws of Behavior Change]]. [Make a habit of trying safe, low-stakes experiments, to learn more about yourself & what works for you](https://blog.ncase.me/30). They can also cause addictions and be harmful to us. It's important to be aware of your habits and know how to break and build them. Habits are the compound interest of self-improvement. Make small habits that push you forward, little by little, and make them [compound](https://www.investopedia.com/terms/c/compoundinterest.asp). Remember, there are [no hacks](http://www.collaborativefund.com/blog/useful-hacks/) in life! @@ -12,11 +12,11 @@ They can also cause addictions and be harmful to us. It's important to be aware Helpful steps for building habits: -- **Build a [[routine]]**. Set a specific, repeating time when you will do the thing in your calendar. Keep that slot clear, always. Never let something interrupt this task. Shape the environment to support your routine and habits. Make it easy to start and make it hard or impossible to not do it. If you don't have [[systems]], your environment determines your actions. Replace self-control with [[Planning]] and proper environment (uninstalling a game removes a future decision to play it). You can also **stack habits**. Connect the habit that doesn't appear that much to you to what you strongly like. Then you'll be doing it for what you like. +- **Build a [[routine]]**. Set a specific, repeating [[Time|time]] when you will do the thing in your calendar. Keep that slot clear, always. Never let something interrupt this task. Shape the environment to support your routine and habits. Make it easy to start and make it hard or impossible to not do it. If you don't have [[systems]], your environment determines your actions. Replace self-control with [[Planning]] and proper environment (uninstalling a game removes a [[Future|future]] decision to play it). You can also **stack habits**. Connect the habit that doesn't appear that much to you to what you strongly like. Then you'll be doing it for what you like. > If you want to change a river's path, you don't place a sign with the direction in which you want water to go, you dig holes so water flows into them. - **Learn to say no**. If someone wants you to do something else during this time slot, say no, and tell them why. - **Never break the routine**. Breaking it once makes it much easier to break the next scheduled time. Instead, think about the smallest thing you can perform and make it your goal for the day. If you do break it, feel bad about it and get back on the horse as fast as you can. Use forcing functions to your advantage. A forcing function is any task, activity or event that forces you to take action and produce a result. Have a bias toward taking action. No lazy deferring. When it's habit time, it should require more willpower to break the habit than to keep going. [Become a mindless robot and don't think twice](http://web.archive.org/web/20230504021809/https://tiffanymatthe.com/not-thinking). - - Make the decision in advance. E.g. the two-minute rule, where you decide that for anything that takes less than two minutes, you do it. No thinking, no arguing, just swift action. Another method is planning out your days in advance. Then when the time comes, you can mindlessly follow the schedule you have made for yourself. + - Make the decision in advance. E.g. the two-minute rule, where you decide that for anything that takes less than two minutes, you do it. No [[Thinking|thinking]], no arguing, just swift action. Another method is planning out your days in advance. Then when the time comes, you can mindlessly follow the schedule you have made for yourself. - Be proactively lazy by organizing a space for its intended purpose. That way you are priming it to make the next action easy. - **Use the power of accountability to reinforce the routine**. If you can find someone who will hold you accountable, do it. Someone who does the routine with you, or a coach who will call you out if you make excuses. Whatever you value, hang around people who are better at it than you are. - **One thing at a time**. Don't build a big routine of 15 tasks at once. Ease into it one habit at a time. Changing one routine at a time allows you to isolate the benefits. Remove related bad habits at the same time. diff --git a/Hobbies.md b/Hobbies.md index 3ed849d..32bea9b 100644 --- a/Hobbies.md +++ b/Hobbies.md @@ -2,12 +2,12 @@ - Playing Minecraft as an excuse to decorate things with [[Pixel Art]] - Creating silly songs with a [[Piano]] -- Listening to lots of [[Podcasts]] +- [[Listening]] to lots of [[Podcasts]] - Following lots of [[YouTube Channels]] - Enjoying [[Science Fiction Reads]] - Curating [[Personal Handbooks]] and [[Company Handbooks]] - Exploring [[Web Based Tools]] -- Learning more about [[Science]] +- [[Learning]] more about [[Science]] - Following the development around [[Open Source Data Projects]] - Playing [[Dungeons and Dragons]] - Participating in [[Datathons]] diff --git a/House.md b/House.md index 9f82161..7a0eadb 100644 --- a/House.md +++ b/House.md @@ -2,7 +2,7 @@ - [Living in the same place as the people you love matters](https://waitbutwhy.com/2015/12/the-tail-end.html). - As with many other things, there are [patterns that will help you structure it](http://web.archive.org/web/20230411022833/http://www.architectureweek.com/topics/patterns.html). -- Any time you have a problem, search for a solution. +- Any time you have a [[Problem Solving|problem]], search for a solution. - Be pro-active about modifying your environment. You can drastically improve your environment for usually little effort and money. ## Buying @@ -14,7 +14,7 @@ - Good illumination (daylight has a strong effect on mood) and a view of natural beauty (nature increases mental functioning and concentration) aren't just pleasant luxuries, but can make important practical differences in your [[Health]]. Light and plants make a difference. - Aim for a small sunny place in the winter! - Aim for a place where you can set up a fire. Humans have spent most of their history around fire sharing stories. -- Research nearby future facilities. Schools, shopping malls, coffee shops, transportation, ... +- Research nearby [[Future|future]] facilities. Schools, shopping malls, coffee shops, transportation, ... - Check the electrical and water installations of the building. - Treat it as a [[Finances|financial]] asset. Think about how much it'll sell for. diff --git a/How To Do Hard Things.md b/How To Do Hard Things.md index 080bd9a..e3a9d4e 100644 --- a/How To Do Hard Things.md +++ b/How To Do Hard Things.md @@ -11,7 +11,7 @@ Useful when you know what success looks like but cannot currently achieve it. 3. Do the modified thing until it is no longer hard. 4. If you get stuck, do one of the following: - Go back to step 3 and pick a different way in which the problem is hard. - - Recursively apply the general system for learning to do hard things to the thing you're stuck on. + - Recursively apply the general system for [[Learning|learning]] to do hard things to the thing you're stuck on. - Go ask an expert or a rubber duck for advice. - If you're still stuck after trying the first three, it's possible that you may have hit some sort of natural difficulty limit and may not be able to make progress. 5. If the original hard thing is now easy, you're done. If not, go back to step 2. diff --git a/IPFS.md b/IPFS.md index 31ec5de..46595a0 100644 --- a/IPFS.md +++ b/IPFS.md @@ -6,9 +6,9 @@ - Once something is added, it can't be changed anymore. - At the heart of everything is the concept of content identifiers (CIDs). - A CID isn't just a hash of a file—it's a combination of the hash, metadata about how the content is hashed, details about encoding, and instructions for interpreting the data. - - They provide portability and openness. + - They provide portability and [[Openness|openness]]. - Keeping files available is a challenge. If the nodes storing a file go down, it'll disappear from the network. - - Filecoin's idea was to help with this by adding incentives to the equation. + - Filecoin's idea was to help with this by adding [[Incentives|incentives]] to the equation. - [Content addressing enables robust data integrity checks and efficient networking: systems can verify they received exactly what they asked for and avoid downloading the same content twice. The linked data part lets you link to stuff by its hash. You can build very big graphs with these primitives](https://dasl.ing/). ## [IPLD](https://ipld.io/) @@ -16,8 +16,8 @@ - Data model for IPFS - Everything is a node. Nodes have types. - The data structure is a Merkle Tree. -- Makes it easy to have interoperability of data. Useful for distributed databases. - - Bridges content addressing and distributed systems. +- Makes it easy to have interoperability of data. Useful for distributed [[Databases|databases]]. + - Bridges content addressing and distributed [[Systems|systems]]. ## LibP2P diff --git a/Idempotence.md b/Idempotence.md index 8ec9d84..7525fb3 100644 --- a/Idempotence.md +++ b/Idempotence.md @@ -1,3 +1,3 @@ # Idempotence -[Idempotence](https://en.wikipedia.org/wiki/Idempotence) is the property of a process that when run 1 or more times, it only has the effect of being run once. An idempotent function is deterministic and repeatable. +[Idempotence](https://en.wikipedia.org/wiki/Idempotence) is the property of a [[Processes|process]] that when run 1 or more times, it only has the effect of being run once. An idempotent function is deterministic and repeatable. diff --git a/Impact Evaluators.md b/Impact Evaluators.md index b40ec1d..f073a4b 100644 --- a/Impact Evaluators.md +++ b/Impact Evaluators.md @@ -24,12 +24,12 @@ It's hard to do [[Public Goods Funding]], open-source software, research, etc. t - Small communities are good for governance, but still need enough sample size / power if you want to make causal claims. - Have a deadline or something like that so it fades away if it's not working or actively used. - [The McNamara Fallacy](https://en.wikipedia.org/wiki/McNamara_fallacy). Never choose metrics on the basis of what is easily measurable over what is meaningful. Data is inherently objectifying and naturally reduces complex conceptions and processes into coarse representations. There's a certain fetish for data that can be quantified. - - Cultivate a culture which welcomes experimentation. + - Cultivate a [[Culture|culture]] which welcomes [[Experimentation|experimentation]]. - Ostrom's Law. "A resource arrangement that works in practice can work in theory" - - Even in environments with clear and easy-to-get metrics, someone has to make a decision about why that metric results in a better allocation instead of another similar metric or a combination. It's all human judgment / governance! + - Even in environments with clear and easy-to-get metrics, someone has to make a decision about why that metric results in a better allocation instead of another similar metric or a combination. It's all human judgment / [[Governance|governance]]! - There is no way around defining a metric/loss function to evaluate allocations! Discussions should be focused on this instead of the specifics of an allocation mechanism. - **Community Feedback Mechanism**. - - Implement robust feedback systems that allow participants to report and address concerns about the integrity of the metrics or behaviors in the community. + - Implement robust [[Feedback|feedback]] systems that allow participants to report and address concerns about the integrity of the metrics or behaviors in the community. - Don't treat evaluation as a black box. Combine impact estimates with [monitoring data, process evaluation, and qualitative feedback](https://openknowledge.worldbank.org/entities/publication/ebbe3565-69ff-5fe2-b65d-11329cf45293) to explain why results happened. - Use the feedback to refine and improve the system. - Prioritize consent and community feedback. @@ -54,7 +54,7 @@ It's hard to do [[Public Goods Funding]], open-source software, research, etc. t - Don't write specific people or outcomes into the mechanism (e.g: using multiple accounts). - **Build anti-Goodhart resilience**. - Any metric used for decisions [becomes subject to gaming pressures](https://en.wikipedia.org/wiki/Campbell%27s_law). - - Design for evolution: + - Design for [[Evolution|evolution]]: - Run multiple evaluation algorithms in parallel and let humans choose. - Use exploration/exploitation trade-offs (like multi-armed bandits) to test new metrics. - Make the meta-layer for evaluating evaluators explicit. @@ -65,14 +65,14 @@ It's hard to do [[Public Goods Funding]], open-source software, research, etc. t - Any mechanism helping under-coordinated parties will also help [over-coordinated parties extract value](https://vitalik.eth.limo/general/2019/04/03/collusion.html). Countermeasures include: - Identity-free incentives (like proof-of-work). - Fork-and-exit rights for minorities. - - Privacy pools that exclude provably malicious actors. + - [[Privacy|Privacy]] pools that exclude provably malicious actors. - Multiple independent "dashboard organizations" preventing capture. - They should be flexible as it's hard to predict ways the evaluation metrics will be gamed. - [Campbell's Law](https://en.wikipedia.org/wiki/Campbell%27s_law). The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. - **Separate data from judgment**. Impact Evaluators work like [[Data Culture|data-driven organizations]]: - Gather objective attestations about work (commits, usage stats, dependencies). - Apply multiple "evaluation lenses" to interpret the data. - - Let funders choose which lenses align with their values. + - Let funders choose which lenses align with their [[Values|values]]. - Prefer [[Pairwise Comparisons]] for human input over absolute scoring. Standard methods turn pairs into scores/ranks and handle noisy raters. - Pairwise comparisons are useful to abstract "impact" and other complex and subjective metrics. Makes the decision more local and easier. More of a hunch-based decision. - [Data is good at providing comprehensive coverage of things that are countable. Data is bad at dealing with nuances and qualitative concepts that experts intuitively understand.](https://gov.optimism.io/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239) @@ -161,13 +161,13 @@ It's hard to do [[Public Goods Funding]], open-source software, research, etc. t ## Related Fields -- Reinforcement Learning +- Reinforcement [[Learning]] - Cybernetics -- Game Design +- [[Game Design]] - Social Choice Theory - [[Mechanism Design]] - Computational Social Choice -- Machine Learning +- [[Machine Learning]] - Voting Theory - Process Control Theory - Large Language Models Evaluation diff --git a/Incentives.md b/Incentives.md index 8779aca..babb5e5 100644 --- a/Incentives.md +++ b/Incentives.md @@ -1,6 +1,6 @@ # Incentives -Incentives matter. Everything around you is driven by incentives. There are rarely true "irrational" decisions. [If a decision looks irrational to you, it's most likely because you don't truly understand the incentives driving that person](https://twitter.com/saranormous/status/1527839900069875713). +Incentives matter. Everything around you is driven by incentives. There are rarely true "irrational" [[Making Decisions|decisions]]. [If a decision looks irrational to you, it's most likely because you don't truly understand the incentives driving that person](https://twitter.com/saranormous/status/1527839900069875713). Behavior is hard to fix. When people say they've learned their lesson, they underestimate how much of their previous mistake was caused by emotions that will return when faced with the same circumstances unless incentives have changed. @@ -17,9 +17,9 @@ Behavior is hard to fix. When people say they've learned their lesson, they unde - [Four components of effectively designed incentives are](https://dhruvmethi.substack.com/p/bureaucracy): - Clear problem statement. - Clear target metric to improve. - - Intentional system design. + - Intentional [[Systems|system design]]. - Commitment to study the metric. -- Human values are highly dimensional. Nudging people in the right direction is hard, especially because nudges usually are very low-dimensional. +- Human [[Values|values]] are highly dimensional. Nudging people in the right direction is hard, especially because nudges usually are very low-dimensional. - [Incentives might be perverse and the effects will be unexpected and contrary to the intentions of their designers](https://en.wikipedia.org/wiki/Perverse_incentive) (e.g. Hacktoberfest spamming Pull Requests on Open Source repositories causing more work for maintainers). ## Incentive Framework diff --git a/Journaling.md b/Journaling.md index 9c15901..d7e0166 100644 --- a/Journaling.md +++ b/Journaling.md @@ -3,8 +3,8 @@ - Journaling is a "keystone" habit. It tightly locks all of your other [[habits]] in place. - Every day at some point, just open up a diary, add the date, then start writing. Write what you did today, and how you are feeling, even if it seems boring. - [Use it as a place to ask yourself questions, and answer them](https://sive.rs/dj). -- It helps build a self-evaluation loop in your life. It gives you the ability to discover what you were thinking and how the decisions turned out. If you don't reflect on your past, any improvements you do in the future will be by chance. - - You're doing this for your future self. Future you will want to look back at this time in your life, and find out what you were actually doing, day-to-day, and how you really felt back then. It will help you make better decisions. +- It helps build a self-evaluation loop in your life. It gives you the ability to discover what you were [[Thinking|thinking]] and how the decisions turned out. If you don't reflect on your past, any improvements you do in the [[Future|future]] will be by chance. + - You're doing this for your future self. Future you will want to look back at this [[Time|time]] in your life, and find out what you were actually doing, day-to-day, and how you really felt back then. It will help you make better decisions. - When you are in an intensely emotional mood, journaling can help you more fully experience and understand those emotions. - [Journal about things, people, or situations for which you are grateful](https://youtu.be/fSwpe8r50_o). Consider including negative situations like avoiding an accident, for instance. **Be specific!** - If you care about your thoughts, keep them. diff --git a/Learning.md b/Learning.md index 54ca09b..56f6bdf 100644 --- a/Learning.md +++ b/Learning.md @@ -1,8 +1,8 @@ # Learning - Learning is a process of [[Conceptual Compression]]. -- Understand the base and build from it. First-principles method. Seek the big picture understanding - focus on concepts not details. -- [Don't delegate understanding](https://stephango.com/understand). When you outsource hard things without learning how they work, you become dependent on systems that profit from extraction and intervention. +- Understand the base and build from it. First-principles method. Seek the big picture understanding - [[Focus|focus]] on concepts not details. +- [Don't delegate understanding](https://stephango.com/understand). When you outsource hard things without learning how they work, you become dependent on [[Systems|systems]] that profit from extraction and intervention. - Read about topics you care about, observe the world around you and keep a beginner's mind (*shoshin*). - Find a mentor if you can and ask them questions. - Don't be afraid to [[Asking Questions|ask a question]] that may sound stupid because 99% of the time everyone else is thinking of the same question and is too embarrassed to ask it. @@ -36,7 +36,7 @@ - Have an Analogy about it. - Visualize it in a Diagram. - Gather Examples. Examples are an amazing way to learn things! - - Examples are an excellent way to resolve lossy information transfer - they're a completely different channel of communication than normal. If nothing else, they serve as an error check. + - Examples are an excellent way to resolve lossy information transfer - they're a completely different channel of [[Communication|communication]] than normal. If nothing else, they serve as an error check. - Examples are a great way to transfer tacit knowledge, without necessarily making it legible - this is what it means to build intuition. - Come up with a Plain-English description of the concept. - Then, dive into the Technical side. diff --git a/Life Areas.md b/Life Areas.md index 8f27e02..6b0de04 100644 --- a/Life Areas.md +++ b/Life Areas.md @@ -7,9 +7,9 @@ 3. Location and possessions. Are you tied to one location? Do you own lots of useless stuff? 4. [[Time]] and [[Finances]]. Do you have savings, investments, or assets? Do you have any debt? Do you know where you spend your money? 5. Career and [[Teamwork]]. Do you enjoy your work? Have you optimized your job? -6. Health and fitness. Do you exercise regularly? What is your overall energy level? What are your major health issues and susceptibilities? +6. [[Health]] and [[Fitness|fitness]]. Do you exercise regularly? What is your overall energy level? What are your major health issues and susceptibilities? 7. Knowledge and [[Learning|education]]. Are you developing your mind and learning new things? What are your talents and skills? -8. [[Communication]]. Are you spreading ideas? Can you communicate effectively? Can you discuss difficult or controversial topics? +8. [[Communication]]. Are you spreading [[Ideas|ideas]]? Can you communicate effectively? Can you discuss difficult or controversial topics? 9. [[Relationships]]. Are you meeting new people? Are you taking care of people you love? Are you maintaining your friendships? 10. Emotions. What is your general feeling about life? Are you aware of your emotions as they are happening? How do you rate in terms of integrity, honesty, courage, etc.? 11. [[Productivity]] and organization. Do you act effectively? Are you organized? When you need a piece of information, how easily can you find it? diff --git a/Listening.md b/Listening.md index d25c35f..75205e4 100644 --- a/Listening.md +++ b/Listening.md @@ -3,9 +3,9 @@ - [Become interested in other people](https://www.lesswrong.com/posts/4K5pJnKBGkqqTbyxx/to-listen-well-get-curious). Encourage others to talk about themselves. The more you are interested in others, the more interesting they find you. To be interesting, be interested. Connect with people, don't perform for them. - Being able to listen well is a superpower. Listening is difficult because it involves suppressing your ego long enough to consider what is being said before you respond. - Forget your agenda. Don't have an internal monologue. -- Avoid trying to respond immediately and allow the other person time to finish speaking. +- Avoid trying to respond immediately and allow the other person [[Time|time]] to finish speaking. - Don't answer your phone while speaking with someone in real life. If the real-life conversation starts, it has precedence over the asynchronous ones. - While listening to someone sharing a problem keep asking them "Is there more?", until there is no more. Create space for the other person to continue [[Talking]]. This is called [reflecting](https://programs.clearerthinking.org/become_a_great_listener.html) and simply means repeating back a word or phrase to encourage the other person to go on. -- Use open questions more. Open questions are any questions that cannot be answered with a simple _yes_ or _no_ reply. +- Use [[Open Questions|open questions]] more. Open questions are any questions that cannot be answered with a simple _yes_ or _no_ reply. - When you are feeling confused, do a small summary to make sure you didn't miss anything important. - Giving someone an opportunity to clarify their emotions and thoughts, or to delve deeper into a certain topic, is one way you can help them process and understand their feelings about a situation. diff --git a/Machine Learning.md b/Machine Learning.md index bcb68e6..9165bf5 100644 --- a/Machine Learning.md +++ b/Machine Learning.md @@ -2,16 +2,16 @@ ## Production Project Checklist -1. Frame the problem. Define a clear and concise objective with clear metrics. [Write it as a design doc](https://applyingml.com/resources/ml-design-docs/). To know "what is good enough" you have to collect and annotate more data than most people and organizations want to do. +1. Frame the problem. Define a clear and concise objective with clear [[Metrics|metrics]]. [Write it as a design doc](https://applyingml.com/resources/ml-design-docs/). To know "what is good enough" you have to collect and annotate more data than most people and [[Organizations|organizations]] want to do. 1. Get the data. Make the data tidy. Machine learning models are only as reliable as the data used to train them. [The data matters more than the model](https://twitter.com/beeonaposy/status/1353735905962577920). Data matters more than the model. [The main bottleneck is collecting enough high-quality data and getting it properly annotated and verified](https://news.ycombinator.com/item?id=45875618). Then doing proper evals with humans in the loop to get it right. 1. Explore the data. Verify any assumptions. Garbage in, garbage out. Remove ALL friction from looking at data. 1. Create a model. [Start with the simplest model!](https://developers.google.com/machine-learning/guides/rules-of-ml/). That will be the [baseline model](https://blog.insightdatascience.com/always-start-with-a-stupid-model-no-exceptions-3a22314b9aaa). Evaluate the model with the defined metric. 1. Make sure everything works end to end. _You design it, you train it, you deploy it_. [Deploy the model quickly](https://nlathia.github.io/2019/08/Machine-learning-faster.html) and automatically. Add a clear description of the model. [Monitor model performance in production](https://youtu.be/hqxQO7MoQIE). 1. Make results (models, analysis, graphs, ...) reproducible (code, environment and data). Version your code, data and configuration. Make feature dependencies explicit in the code. Separate code from configuration. -1. Test every part of the system ([ML Test Score](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/aad9f93b86b7addfea4c419b9100c6cdd26cacea.pdf)): data (distributions, unexpected values, biases, ...). +1. Test every part of the [[Systems|system]] ([ML Test Score](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/aad9f93b86b7addfea4c419b9100c6cdd26cacea.pdf)): data (distributions, unexpected values, biases, ...). 1. Iterate. Deliver value first, then iterate. Go back to the first point and change one thing at a time. Machine Learning progress is nonlinear. It's really hard to tell in advance what's hard and what's easy. - [Engineering projects generally move forward, but machine learning projects can completely stall. It's possible, even common, for a week spent on modeling data to result in no improvement whatsoever](https://medium.com/@l2k/why-are-machine-learning-projects-so-hard-to-manage-8e9b9cf49641). - - Track every experiment you do. Keep a reverse-time sorted doc where you add bullet points of what ideas you've tried and how they've gone. Look for data flywheel, harnessing the power of user-generated data to rapidly improve the whole system. These are powerful [[Feedback Loops]]. Attempt a portfolio of approaches. + - Track every experiment you do. Keep a reverse-[[Time|time]] sorted doc where you add bullet points of what [[Ideas|ideas]] you've tried and how they've gone. Look for data flywheel, harnessing the power of user-generated data to rapidly improve the whole system. These are powerful [[Feedback Loops]]. Attempt a portfolio of approaches. 1. Explain your results in terms your audience cares about. [[Data Culture|Data is only useful as long as it's being used]]. These points are expanded with more details in courses like [Made With ML](https://madewithml.com/). @@ -20,19 +20,19 @@ These points are expanded with more details in courses like [Made With ML](https > Don't hope for "great", specify it, measure it, and improve toward it! -- Evals make fuzzy goals and abstract ideas specific and explicit. They help you systematically measure and improve a system. +- Evals make fuzzy [[Goals|goals]] and abstract ideas specific and explicit. They help you systematically measure and improve a system. - Evals are a key set of tools and methods to measure and improve the ability of an AI system to meet expectations. -- [Success with AI hinges on how fast you can iterate](https://hamel.dev/blog/posts/evals/#iterating-quickly-success). You must have processes and tools for evaluating quality (tests), debugging issues (logging, inspecting data), and changing the behavior or the system (prompt eng, fine-tuning, writing code). +- [Success with AI hinges on how fast you can iterate](https://hamel.dev/blog/posts/evals/#iterating-quickly-success). You must have [[Processes|processes]] and tools for evaluating quality (tests), debugging issues (logging, inspecting data), and changing the behavior or the system (prompt eng, fine-tuning, writing code). - Collecting good evals will make you understand the problem better. - Working with probabilistic systems requires new kinds of measurement and deeper consideration of trade-offs. - Don't work if you cannot define what "great" means for your use case. - [Evals replace LGTM-vibes development](https://newsletter.pragmaticengineer.com/p/evals). They systematize quality when outputs are non-deterministic. -- [Error analysis](https://youtu.be/ORrStCArmP4) workflow: build a simple trace viewer, review ~100 traces, annotate the first upstream failure ([open coding](https://shribe.eu/open-coding/)), cluster into themes ([axial coding](https://delvetool.com/blog/openaxialselective)), and use counts to prioritize. Bootstrap with grounded synthetic data if real data is thin. +- [Error analysis](https://youtu.be/ORrStCArmP4) workflow: build a simple trace viewer, review ~100 traces, annotate the first upstream failure ([open coding](https://shribe.eu/open-coding/)), cluster into [[Themes|themes]] ([axial coding](https://delvetool.com/blog/openaxialselective)), and use counts to prioritize. Bootstrap with grounded synthetic data if real data is thin. - [Evals are fundamentally "data science"](https://youtu.be/lA4MfpgF91Y). **Look at your data**, conduct experiments and measure where appropriate, and iterate metrics and approaches. - Pick the right evaluator: code-based assertions for deterministic failures; LLM-as-judge for subjective ones. Keep labels binary (PASS/FAIL) with human critiques. Partition data so the judge cannot memorize answers; validate the judge against human labels (TPR/TNR) before trusting it. - Run evals in CI/CD and keep monitoring with production data. - [Analyze → measure → improve → automate → repeat](https://newsletter.pragmaticengineer.com/p/evals). -- Good eval metrics: +- Good eval [[Metrics|metrics]]: - Measure an error you've observed. - Relates to a non-trivial issue you will iterate on. - Are scoped to a specific failure. diff --git a/Making Decisions.md b/Making Decisions.md index ffe4348..d5a9e2d 100644 --- a/Making Decisions.md +++ b/Making Decisions.md @@ -39,7 +39,7 @@ Decision-making is the process we use to identify and choose alternatives, produ - Most experts aren't communicators and most communicators aren't experts. This often results in research on issues being spun with a narrative by the time it reaches the public. - Beware of cases where the decision lies in the hands of people who would gain little personally, or lose out personally, if they did what was necessary to help someone else. They have no skin in the game and no incentives. - [The decision-makers shouldn't have competing incentives with the decision outcome](https://www.youtube.com/watch?v=Rwxkqno1PTc). -- You have a plan. A time-traveler from the future appears and tells you your plan failed. Which part of your plan do you think is the one that fails? Fix that part. +- You have a plan. A time-traveler from the [[Future|future]] appears and tells you your plan failed. Which part of your plan do you think is the one that fails? Fix that part. - When consensus doesn't occur it's because there isn't a clear answer or because there is a conflict between groups. In these situations, it's up to management to make a decision so the organization can move forward. - If you're in between two decisions, don't half-ass both of them! Do one 100%, then do the other 100%. - People reason more wisely about other people's problems than about their own. diff --git a/Mechanism Design.md b/Mechanism Design.md index e8c9c37..6331703 100644 --- a/Mechanism Design.md +++ b/Mechanism Design.md @@ -2,7 +2,7 @@ Mechanism design is the study of how incentives are created to achieve desired outcomes. It focuses on the design of [[Systems]] and [[Processes]] to achieve desired outcomes. [Mechanisms are algorithms plus incentives](https://balajis.com/p/credible-neutrality). -A mechanism is a tool that takes in inputs from multiple people, and uses these inputs as a way to determine things about its participants' values, so as to make some kind of decision that people care about. In a well-functioning mechanism, the decision made by the mechanism is both efficient - in the sense that the decision is the best possible outcome given the participants’ preferences, and incentive-compatible, meaning that people have the incentive to participate "honestly". +A mechanism is a tool that takes in inputs from multiple people, and uses these inputs as a way to determine things about its participants' [[Values|values]], so as to make some kind of decision that people care about. In a well-functioning mechanism, the decision made by the mechanism is both efficient - in the sense that the decision is the best possible outcome given the participants’ preferences, and incentive-compatible, meaning that people have the incentive to participate "honestly". - Software is eating Mechanism Design. Incentives can be encoded in [[blockchain|blockchains]]. - The simpler a mechanism is, and the fewer parameters a mechanism has, the less space there is to insert hidden privilege for or against a targeted group. If a mechanism has fifty parameters that interact in complicated ways, then it's likely that for any desired outcome you can find parameters that will achieve that outcome. @@ -12,13 +12,13 @@ A mechanism is a tool that takes in inputs from multiple people, and uses these - Mechanism design flips game theory: choose rules (outcomes & payments) so strategic agents reach desired outcomes. - An agent's "type" is their private information that determines how much they value each possible outcome (e.g: a bidder's valuation for an item). - In quasilinear settings (utility = value − payment), mechanisms map reported types to decisions and transfers. -- The Revelation Principle lets us focus on direct, truth-telling mechanisms: DSIC (dominant strategies) or BIC (Bayes-Nash). +- The Revelation Principle lets us [[Focus|focus]] on direct, truth-telling mechanisms: DSIC (dominant strategies) or BIC (Bayes-Nash). - Gibbard–Satterthwaite impossibility: with three or more options and unrestricted preferences, only dictatorial DSIC choice functions exist. - Top Trading Cycles yields Pareto-efficient, individually rational, strategyproof allocations in exchange problems. - Most bits of information in the output of a mechanism should come from the participants' inputs, not from hard-coded rules inside of the mechanism itself. - A good mechanism is also a mechanism that actually does solve the problems that we care about. If it can't be done completely neutrally, it doesn't mean it should not be done at all. - Any mechanism that can help genuinely under-coordinated parties coordinate will, without the right safeguards, also help already-coordinated parties (such as many accounts controlled by the same person) [over-coordinate](https://vitalik.eth.limo/general/2019/04/03/collusion.html) with potential ways to "do wrong" (e.g: extract money from the system). -- You can increase mechanism complexity if you trade it off for identity or collusion resistance. If you figure out a way to make the mechanism identity-resistant, it'll support more complex setups. +- You can increase mechanism complexity if you trade it off for [[Identity|identity]] or collusion resistance. If you figure out a way to make the mechanism identity-resistant, it'll support more complex setups. - [Truthtelling games](https://jonathanwarden.com/truthtelling-games/) can incentivize honesty through coordination games where participants win by giving the same answer as others, with truth serving as a powerful Schelling point (truthtelling is the winning strategy only if everybody else tells the truth). Information elicitation mechanisms can get people to reveal private/subjective information truthfully even without verification. - Some of the interesting properties of a mechanism are: local/bottom-up decision-making, can be combined in different layers (horizontal / vertical), and [[Modularity|modularity]]. - Mechanisms should preserve human agency instead of replacing it. @@ -37,7 +37,7 @@ A mechanism is a tool that takes in inputs from multiple people, and uses these - Auctions. The input is bids, the output is who gets the item being sold, and how much the buyer must pay. - [Vickrey–Clarke–Groves auction](https://en.wikipedia.org/wiki/Vickrey%E2%80%93Clarke%E2%80%93Groves_auction). - [Second-price auction](https://en.wikipedia.org/wiki/Generalized_second-price_auction). -- Quadratic voting and funding as a way of coming to agreement on matters of governance and [[Public Goods Funding]]. +- Quadratic voting and funding as a way of coming to agreement on matters of [[Governance|governance]] and [[Public Goods Funding]]. ### Resources @@ -56,7 +56,7 @@ A mechanism is a tool that takes in inputs from multiple people, and uses these - **Collusion-safe games**. Rely on identity-free incentives (PoW-like) or security-deposit futarchy where bad coordination is personally costly. - **Fork-and-exit**. Make systems easy to split so minority users can counter-coordinate against cartels. - **Quadratic Mechanisms**. [Funding](https://vitalik.eth.limo/general/2019/12/07/quadratic.html) and voting that make influence proportional to square root of resources, reducing plutocracy. -- **Prediction and Decision Markets (Futarchy)**. ["Vote values, bet beliefs"](https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c) - conditional markets choose policies that maximize agreed-upon metrics. +- **Prediction and Decision Markets (Futarchy)**. ["Vote values, bet beliefs"](https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c) - conditional markets choose policies that maximize agreed-upon [[Metrics|metrics]]. - **Distilled-Human-Judgment Markets**. A jury scores a small sample, open AI/human traders supply full answers, rewards fit; scales expertise cheaply. - **Engine-and-steering-wheel pattern**. Open competition of AI "engines" acts under a simple, credibly neutral rule set and is audited/reinforced by humans. - **Research-Augmented Bonding Curves (ABCs) / Curation Markets**. Automated market makers that route fees to upstream dependencies based on usage. @@ -65,7 +65,7 @@ A mechanism is a tool that takes in inputs from multiple people, and uses these - **Deliberative protocols**. [Structured discussion processes](https://jonathanwarden.com/deliberative-consensus-protocols/) that surface information before voting. - **Harberger Taxes/COST (Common Ownership Self-assessed Tax)** - Entities self-assess value and pay tax on it, but must sell at that price if someone wants to buy. Useful for allocating scarce positions/rights in evaluation systems. - **Dominant Assurance Contracts** - Entrepreneur provides refund + bonus if funding threshold isn't met, solving the assurance problem in public goods funding more elegantly than traditional crowdfunding. -- **Conviction Voting** - Preferences gain strength over time rather than snapshot voting. Voters continuously express preferences and conviction builds, reducing governance attacks. +- **Conviction Voting** - Preferences gain strength over [[Time|time]] rather than snapshot voting. Voters continuously express preferences and conviction builds, reducing governance attacks. - **Retroactive Oracles** - Designated future evaluators whose preferences are predicted by current markets. Separates the "who decides" from "what they'll value" questions. - **Sortition/Random Selection** - Randomly selected evaluation committees from qualified pools. Reduces corruption and strategic behavior while maintaining statistical representativeness. - **Optimistic Mechanisms** - Actions are allowed by default but can be challenged within a time window. Reduces friction for honest actors while maintaining security. @@ -75,8 +75,8 @@ A mechanism is a tool that takes in inputs from multiple people, and uses these - **Threshold Cryptography/Secret Sharing** - For private evaluation scores that only become public when aggregated. Prevents anchoring and collusion during evaluation. - **Augmented Bonding Curves with Vesting** - Time-locked rewards that vest based on continued positive evaluation over time, aligning long-term incentives. - **Multi-armed Bandits** - Adaptive mechanism selection algorithms that balance exploration and exploitation. Dynamically choose between evaluation mechanisms based on historical performance and context to optimize for both learning and effectiveness. -- **Privacy Pools** - Systems that maintain participant privacy while excluding provably malicious actors. Allow honest participants to prove non-membership in bad actor sets without revealing their identity. -- **Reinforcement Learning for Meta-Evaluation** - Use RL to evolve evaluation mechanisms through trial and error. The system learns which evaluation approaches work best in different contexts by treating mechanism selection as a sequential decision problem. +- **Privacy Pools** - Systems that maintain participant [[Privacy|privacy]] while excluding provably malicious actors. Allow honest participants to prove non-membership in bad actor sets without revealing their identity. +- **Reinforcement Learning for Meta-Evaluation** - Use RL to evolve evaluation mechanisms through trial and error. The system [[Learning|learns]] which evaluation approaches work best in different contexts by treating mechanism selection as a sequential decision problem. - **Genetic Algorithms** - Evolution-based optimization for evaluation mechanisms. Breed and mutate successful evaluation strategies, allowing the system to discover novel approaches through recombination and selection pressure. - **Schelling Point Coordination Games** - Information elicitation mechanisms where truth naturally emerges as the coordination point. Participants are incentivized to report honestly because they expect others to do the same, making truth the natural focal point. diff --git a/Meditation.md b/Meditation.md index f11034d..d30ab5f 100644 --- a/Meditation.md +++ b/Meditation.md @@ -1,7 +1,7 @@ # Meditation - Meditation is a simple practice available to all, which can reduce stress, increase calmness and clarity and promote [[Mental Health|happiness]]. Learning how to meditate is straightforward, and the benefits can come quickly. Even just a few minutes a day can make a big difference revealing and refactoring various [mental motions](https://www.lesswrong.com/posts/WYmmC3W6ZNhEgAmWG/a-mechanistic-model-of-meditation#Uses_for_moments_of_introspective_awareness_). - - If you have more introspective awareness of the exact processes that are happening in your mind, you can make more implicit assumptions conscious, causing your brain's built-in contradiction detector to notice when they contradict your later learning. Also, getting more feedback about what exactly is happening in your mind allows you to notice more wasted motion in general. + - If you have more introspective awareness of the exact [[Processes|processes]] that are happening in your mind, you can make more implicit assumptions conscious, causing your brain's built-in contradiction detector to notice when they contradict your later [[Learning|learning]]. Also, getting more [[Feedback|feedback]] about what exactly is happening in your mind allows you to notice more wasted motion in general. - The goal of meditation is not to quit having thoughts, but to observe them, watch them without reacting and rationalizing them. This helps rewire the way your brain works similar to cognitive behavioral therapy. Typically, people who work towards this, have less anxiety and depression because they don't react to every thought they have, which gives them a greater control of their emotions. - In [[programming]] terms, it defragments the hard drive and repairs errors in the OS. - Too many distractions lead to a heavy mind. diff --git a/Meetings.md b/Meetings.md index cb9645a..7a7d74e 100644 --- a/Meetings.md +++ b/Meetings.md @@ -32,7 +32,7 @@ Meetings are essential for brainstorming and group decision-making, but they oft - Only attend meetings if genuinely interested and can contribute. - Prepare in advance by reviewing agenda and related materials. -- Actively participate in discussions rather than passively listening. +- Actively participate in discussions rather than passively [[Listening|listening]]. - Decline meetings when appropriate - it's better than attending without engagement. - Read meeting minutes if not attending to stay informed. @@ -49,12 +49,12 @@ Meetings are essential for brainstorming and group decision-making, but they oft - How the product is being received in the market. - How the most important customers are succeeding (or not) using the product. - How the team is performing and any people changes needed. -- Financial position of the company and review of metrics. +- Financial position of the company and review of [[Metrics|metrics]]. ## Meeting Culture Recommendations - Prioritize written [[Communication]] for knowledge sharing over meetings. - Make it culturally acceptable to enter/leave meetings strategically. - Create a culture where declining meetings without guilt is normal. -- Focus on creating intentional, purposeful meetings rather than habitual ones. +- [[Focus]] on creating intentional, purposeful meetings rather than habitual ones. - "The goal of an exceptional meeting culture is to allow for people to constructively decline meetings by fully understanding the consequences of their action." diff --git a/Mental Health.md b/Mental Health.md index 8f01a8b..174082c 100644 --- a/Mental Health.md +++ b/Mental Health.md @@ -17,10 +17,10 @@ Aversions can be conscious or unconscious, reasoned or felt, verbal or visceral, There are many reasons folks feel [helpless blankness](https://www.lesswrong.com/posts/puhPJimawPuNZ5wAR/learned-blankness) about understanding a given topic, including: -1. Simple habit. You are not used to thinking about it; and so you just automatically don't. -2. Desire to avoid initial blunders that will force you to emotionally confront potential incompetence (as with my fear of writing fiction). +1. Simple habit. You are not used to [[Thinking|thinking]] about it; and so you just automatically don't. +2. Desire to avoid initial blunders that will force you to emotionally confront potential incompetence (as with my fear of [[Writing|writing]] fiction). 3. Avoidance of social conflict, or of status-claims; if your boss/friends/whoever will be upset by your disagreement, it may be more comfortable to “[not understand](https://quoteinvestigator.com/2017/11/30/salary)” the domain. -If you'd like to reduce your learned blankness, try to [notice areas you care about, that you have been treating as blank defaults](https://www.lesswrong.com/posts/puhPJimawPuNZ5wAR/learned-blankness#Notice_your_learned_blankness). Then, seed some thoughts in that area: set a timer, and write as many questions as you can about that topic before it beeps. +If you'd like to reduce your learned blankness, try to [notice areas you care about, that you have been treating as blank defaults](https://www.lesswrong.com/posts/puhPJimawPuNZ5wAR/learned-blankness#Notice_your_learned_blankness). Then, seed some thoughts in that area: set a timer, and write as many [[Open Questions|questions]] as you can about that topic before it beeps. Aversions are [[Problem Solving|decomposable problems]]. Break them down into smaller pieces so that you can think about them separately one at a time and solve them. diff --git a/Mental Models.md b/Mental Models.md index 11a71d0..003f263 100644 --- a/Mental Models.md +++ b/Mental Models.md @@ -38,7 +38,7 @@ A list of mental models that I find interesting and useful. List comes from [And ### Perspective & Understanding - Reality tunnel: Your perspective limits what truths you can perceive. -- Bridges: Ideas that connect different states of understanding. +- Bridges: [[Ideas]] that connect different states of understanding. - Congruent bridges: When different paths lead to the same insight. - Categorization errors: The boundaries we draw between concepts are artificial and often misleading. - Transposed Model: Testing ideas by applying them in different contexts. @@ -56,20 +56,20 @@ A list of mental models that I find interesting and useful. List comes from [And - Pattern matching: Expertise comes from recognizing recurring templates. - Training data: Some lessons can only be learned through examples. -- Social learning: How humans learn from each other. +- Social [[Learning|learning]]: How humans learn from each other. - Exploration vs objective-seeking: Sometimes wandering beats having a destination. - Different advice on different days: Context determines what guidance is useful. - Ecology of practice: Balanced collection of practices for overall human development. ### Communication & Information -- Bottlenecked communication: Our ability to express internal experiences is fundamentally limited. +- Bottlenecked [[Communication|communication]]: Our ability to express internal experiences is fundamentally limited. - Controlling inputs: Managing information flow is crucial for clear thinking. - Polluting the information ecology: Some actors degrade our shared ability to make sense of reality. - Steelmanning: Address the strongest possible version of an opponent's argument. - Definitional disagreement: Many arguments stem from different underlying definitions. -### Culture & Ideas +### [[Culture]] & Ideas - Everything is a remix: All creation builds on what came before. - Memetics: Ideas evolve and spread through fitness landscapes. @@ -83,7 +83,7 @@ A list of mental models that I find interesting and useful. List comes from [And - Parts work: The psyche is multi-agent and requires internal coordination. - Resonance: Non-intellectual alignment with ideas or experiences. - Self vs identity: The difference between core experience and chosen labels. -- Short-circuited rewards: When natural feedback loops get hijacked. +- Short-circuited rewards: When natural [[Feedback Loops|feedback loops]] get hijacked. - Placebo pump: Harnessing belief and expectation to produce real effects. - Smells: Intuitive warnings that something may be wrong. diff --git a/Metrics.md b/Metrics.md index 7ba5d84..2f21034 100644 --- a/Metrics.md +++ b/Metrics.md @@ -11,12 +11,12 @@ - Pick the simplest metric that works for you. Metrics definitions should be as easy as a tooltip away to find. - [Metrics are a tool, but not the destination](https://breakingpoint.substack.com/p/you-have-too-many-metrics)! You want to use the fewest metrics possible to cover all the fundamentals of your business. - Pick a metric you want to improve or affect! -- Organizations need three things related to metrics: +- [[Organizations]] need three things related to metrics: 1. A [[Metrics|process for defining metrics]]. 2. A single source of truth for the metrics. 3. A way to get metrics to all systems. - Product metrics allow measuring product progress and creating alignment in an outcome-oriented way. There are many product frameworks available to help us think about the right key things to track. Think about **[product metrics that matter](https://uxdesign.cc/product-metrics-that-matter-951b9e4d4eca)** for you. -- Push a culture of metrics and goals as a source of learning, not promotions or success delegation. +- Push a [[Culture|culture]] of metrics and goals as a source of [[Learning|learning]], not promotions or success delegation. - Vanity metrics are surface-level metrics. They're often large measures, like number of downloads, that impress others. **Clarity metrics** are operational metrics, like the number of minutes a day your product actually gets used or how long it took for a user to get service. These are the hidden gears that drive growth. - [Factors that are easiest to measure or most visible are the ones that get the most attention and might not be the most important ones](https://wiki.c2.com/?SovietShoeFactoryPrinciple). - [When people are pressured to meet a target value there are three ways they can proceed](https://www.goodreads.com/book/show/63859): @@ -27,11 +27,11 @@ 1. A criterion — the thing you want to measure. 2. Test procedure — how will you measure the thing? 3. Decision rule — how will you decide if the thing you're measuring should be included in the count? -- A process is predictable if it just contains routine variation. It is unpredictable if it contains both routine and special variation. You can only improve a process if you first make it predictable. +- A process is predictable if it just contains [[Routine|routine]] variation. It is unpredictable if it contains both routine and special variation. You can only improve a process if you first make it predictable. - Metrics should help form a working causal model of the business you're in so you know what interventions you can make and can predict the consequences. - A metric set should map to a theory of change: input, output, outcome, impact. -## Good Metric Checklist +## Good Metric [[Checklist]] - **Specific and sensitive**: Metrics should be specific to the product or feature, and need to be explicitly and quantitatively defined. The metric should also be sensitive enough to measure the impact we expect to see. - **Robust**: To complement the **sensitivity** criteria above, we also need to make sure the metric is measuring only the effect of the product of interest, and that it is not reactive to things we expect to change but don't control. Related to [internal validity](https://en.wikipedia.org/wiki/Internal_validity), we should try to avoid using a metric that can be significantly influenced by anything other than the product/feature we care about. diff --git a/Mindfulness.md b/Mindfulness.md index 03e4ac4..9b0e78b 100644 --- a/Mindfulness.md +++ b/Mindfulness.md @@ -46,7 +46,7 @@ - Avoid conspicuous consumption. - Overcome the bystander effect: there is something that everyone wants to *happen* but nobody wants to be the one to *do* it. Develop the reflex of noticing bystander apathy in your environment, and actively do the thing. E.g. ask a question when there's a confusing point in a talk, notice tiny tragedies of the commons (an empty jug of water that nobody wants to refill), notice when everyone feels uncomfortable being the first to, say, dance at a party, and just do it. - [Our behavior is made up of a complex and chaotic soup of so many factors that it's downright silly to think there's a singular, autonomous "you" calling the shots](https://youtu.be/GRYcSuyLiJk). -- Tools and ideas are not neutral. They bake in principles and values. +- Tools and [[Ideas|ideas]] are not neutral. They bake in principles and [[Values|values]]. - E.g: [[Social Media Issues|social media]] encourages rage, [[Blockchain]] protocols "waste" energy using Proof of Work. - A person holding a hammer interacts with the world in a different way. It is a different entity. Same with ideas. - Most of the world is held together with duct tape, so don't be surprised when it breaks. @@ -57,5 +57,5 @@ - Seek real [[Feedback]] (e.g. anonymous). - Meet a wide range of people, as valuable opportunities often come from unexpected connections. - Burnout is a major killer of agency and creativity. Prioritize rest and sustainable effort. -- [Style is a set of constraints that you stick to](https://stephango.com/style). Style emerges from consistency, and having a style opens your imagination. Collect constraints you enjoy. Unusual constraints make things more fun. You can always change them later. This is your style, after all. It’s not a life commitment, it’s just the way you do things. Having a style collapses hundreds of future decisions into one, and gives you focus. E.g: always pluralize tags to never have to wonder what to name new tags. +- [Style is a set of constraints that you stick to](https://stephango.com/style). Style emerges from consistency, and having a style opens your imagination. Collect constraints you enjoy. Unusual constraints make things more fun. You can always change them later. This is your style, after all. It’s not a life commitment, it’s just the way you do things. Having a style collapses hundreds of [[Future|future]] decisions into one, and gives you focus. E.g: always pluralize tags to never have to wonder what to name new tags. - [Slowing down, and offering more time to the act of consumption, immediately increases comprehension and enjoyment](https://www.raptitude.com/2025/12/maybe-the-default-settings-are-too-high/). Slowing down your rate of consumption will inevitably change what you want to consume. diff --git a/Modularity.md b/Modularity.md index 5b831c6..7e54438 100644 --- a/Modularity.md +++ b/Modularity.md @@ -4,4 +4,4 @@ Modularity is a property of [[systems]]. It means: - Loosely coupling. Breaking a chair shouldn't break the dinner! - Multiple parts that play well together. This allows systems to develop [[emergence]]. -- Dividing a large system into multiple smaller competing ones will make them evolve different rule sets and will allow you to pick the best one. +- Dividing a large system into multiple smaller competing ones will make them [[Evolution|evolve]] different rule sets and will allow you to pick the best one. diff --git a/NFTs.md b/NFTs.md index 57aa13d..0675a12 100644 --- a/NFTs.md +++ b/NFTs.md @@ -4,5 +4,5 @@ Unique assets whose value is independent of one another. For example, an NFT mig - Today, if you create something digital, ownership is tied in with distribution. If someone makes an `mp3` or a `jpeg` piece of art and posts it or sells it online, you'll have to have it to consume it (DRM can sometimes change this). - With NFTs, the digital asset you create has a clear, provable owner (you!) and anyone you sell it to (and anyone thereafter) can trace it back to you, the creator, and know it is authentic. -- When you create this NFT thing, you can create rules around it via a smart contract. One of these rules could be that anytime this piece of art you made is transferred across owners, you get 5% of what the new person paid for it, in perpetuity, automatically. +- When you create this NFT thing, you can create rules around it via a [[Blockchain|smart contract]]. One of these rules could be that anytime this piece of art you made is transferred across owners, you get 5% of what the new person paid for it, in perpetuity, automatically. - There are [many problems with NFTs](https://youtu.be/YQ_xWvX1n9g)! diff --git a/Network Effects.md b/Network Effects.md index 6b90a98..8b75c62 100644 --- a/Network Effects.md +++ b/Network Effects.md @@ -9,7 +9,7 @@ Network [diffusion](http://web.archive.org/web/20240705113531/https://meltingasp - Infectious diseases jumping from host to host within a population. - Memes spreading across a follower graph on social media. - A wildfire breaking out across a landscape. -- Ideas and practices diffusing through a [[culture]]. +- [[Ideas]] and practices diffusing through a [[culture]]. - Neutrons cascading through a hunk of enriched uranium. - Economies of scale. More scale means more scale. - More customers mean more social proof means more customers. diff --git a/News.md b/News.md index e5a7bb3..e4facc4 100644 --- a/News.md +++ b/News.md @@ -6,7 +6,7 @@ - [[Social Media Issues|Social media and news usually work against fixing the issues]] which most of us agree should be fixed. Instead they focus only on perpetuating the things which keep us all discussing, forever. - [The amygdala (emotional core of the mind) is built to make us react to "threatening" information that doesn't fit into our worldview the same way we react to a predator. That's called the "Backfire Effect"](https://theoatmeal.com/comics/believe). Remember that [[Thinking|your worldview is not perfect]]! - If our perceptions of reality are increasingly informed by media with other-than-truth motivations, we'll increasingly lose our handle on the truth. [We will be living in a state of split consciousness](https://sandhoefner.com/2019/01/27/on-disbelieving-atrocities/), being immune to thousands of deaths and concerned about the latest changes to your favorite team. -- We're losing our ability to think together. Communities can only think when people talk and when they're free to say what they really think. As echo chambers grow larger and more intimidating, people inside them are afraid to defy the sacred narrative. And the more all-encompassing [[Politics|political]] identities become, the more topics turn from kickable machines to precious infants. Meanwhile, inter-group communication suffers even more, as opposing groups become totally unable to collaborate on [[ideas]]. +- We're losing our ability to think together. Communities can only think when people [[Talking|talk]] and when they're free to say what they really think. As echo chambers grow larger and more intimidating, people inside them are afraid to defy the sacred narrative. And the more all-encompassing [[Politics|political]] identities become, the more topics turn from kickable machines to precious infants. Meanwhile, inter-group [[Communication|communication]] suffers even more, as opposing groups become totally unable to collaborate on [[ideas]]. - A polarized society that isn't capable of building broad coalitions can't take forward steps—it can only self-inflict. - [A large amount of stuff in the news is flat-out wrong](https://www.experimental-history.com/p/reading-the-news-is-the-new-smoking). - The news also biases the layperson's perception of risk. The very fact that bad events are rare these days makes them newsworthy. With a large sample size, unusual tragedies happen daily, and they end up on the news nightly. For example, car crashes might cause more damage than other events like floods or fires. @@ -16,7 +16,7 @@ - The profit comes from advertising, and advertising revenue is maximized by pulling the largest audience, holding their attention for the longest possible time, and putting them into the mental state most conducive to purchasing the products of the advertisers (which turns out to be helplessness and vulnerability). This is why the news always starts out with a sensationalist take on a topic of at least plausible national interest, takes a detour into truly horrific and depressing irrelevant tragedies, then ends on an uplifting note with something like a defiant entrepreneur or a caring soup kitchen. An emotional roller-coaster ride every day of the week. They don't explore solutions. - [There isn't enough actual news (i.e. events that are "new") to fill the standard news slot — so the fillers became pundits and commentators interpreting "news" and "potential news" for us](https://jjbeshara.com/2018/11/20/the-information-pathology-2/). Humans are good at finding efficiencies, and potential events far outweigh the number of past events, and potential negative events captivate our attention better than potential positive events, so these news cycles naturally became dominated by commentators interpreting any number of potential negative events. - [Progress happens too slowly to notice](http://web.archive.org/web/20220629122132/https://www.collaborativefund.com/blog/lots-of-overnight-tragedies-no-overnight-miracles/), setbacks happen too fast to ignore. Bad things can happen fast, but almost all good things happen slowly. Even if the trendlines are improving, the feeling the news gives is the opposite. -- Online & mainstream media and social networking have become increasingly misleading as to the state of the world by focusing on ‘stories' and ‘events' rather than trends and averages. This is because as the global population increases and the scope of media increases, media's urge for narrative focuses on the most extreme outlier data points—but such data points are, at a global scale, deeply misleading as they are driven by unusual processes such as the mentally ill or hoaxers. +- Online & mainstream media and social networking have become increasingly misleading as to the state of the world by focusing on ‘stories' and ‘events' rather than trends and averages. This is because as the global population increases and the scope of media increases, media's urge for narrative focuses on the most extreme outlier data points—but such data points are, at a global scale, deeply misleading as they are driven by unusual [[Processes|processes]] such as the mentally ill or hoaxers. - [Sturgeon's law](https://en.wikipedia.org/wiki/Sturgeon%27s_law): 90% of everything is crap. News is often "correct" on a basic level, but really more like "yes, but it's complicated" on a deeper level. If you've ever seen a surface-level description of something you know about at a deep level, and you realize how wrong it is, or at least how much nuance it's missing. Realize that it's like that with everything. - [On certain topics, it's good to remember that you're often being informed by the most delusional people](https://twitter.com/waitbutwhy/status/1436006304892559365). - [The news's obsession with having a little bit of information on a wide variety of subjects means that it actually gets most of those subjects wrong](http://www.aaronsw.com/weblog/hatethenews). diff --git a/Nonviolent Communication.md b/Nonviolent Communication.md index e7aeae2..76c0b1d 100644 --- a/Nonviolent Communication.md +++ b/Nonviolent Communication.md @@ -1,10 +1,10 @@ # Nonviolent Communication -[Nonviolent communication (NVC)](https://www.clearerthinking.org/post/2019/03/06/want-to-improve-your-relationships-try-nonviolent-communication-1) is a popular method of conflict resolution that privileges unbiased evidence and specificity. It is also a tool for thinking that may help improve the way you communicate, whether it be with family, friends, or colleagues. +[Nonviolent communication (NVC)](https://www.clearerthinking.org/post/2019/03/06/want-to-improve-your-relationships-try-nonviolent-communication-1) is a popular method of conflict resolution that privileges unbiased evidence and specificity. It is also a tool for [[Thinking|thinking]] that may help improve the way you communicate, whether it be with family, friends, or colleagues. ## The Four Steps 1. **Observe Facts** - observe the specific facts that are affecting our well-being, and bring them up with the other person. -2. **Note Feelings** - introspect about what exactly we are feeling in response to what we've observed, and communicate these feelings. +2. **Note Feelings** - introspect about what exactly we are feeling in response to what we've observed, and [[Communication|communicate]] these feelings. 3. **Uncover Desires** - figure out the desires, wants and [[Values]] that are creating our feelings, and explain them to the other person. 4. **Make Requests** - ask for concrete actions to help resolve the situation. diff --git a/Nutrition.md b/Nutrition.md index f6c3149..7f90ce9 100644 --- a/Nutrition.md +++ b/Nutrition.md @@ -2,7 +2,7 @@ - Your *ideal* diet (and weight) is unique. Everyone's body and nutritional needs are surprisingly different. What works for one person won't work for another. - What [[Cooking|you eat]] is one of the most important things to understand given how much it affects the rest of your life. Diet is probably the most important single factor in your [[health]], body composition and overall appearance. -- Drink more water. Water regulates virtually every bodily process in some way. Drinking more water is a simple, virtually cost-free thing you can do to improve your overall [[health]]. +- Drink more water. Water regulates virtually every bodily [[Processes|process]] in some way. Drinking more water is a simple, virtually cost-free thing you can do to improve your overall [[health]]. - [Fasting can be useful in some situations](https://www.reddit.com/r/productivity/comments/6demoi/why_you_should_try_intermittent_fasting_mood/). - Studies suggest a strong link between calorie restriction and longevity. Caloric restriction stimulates many beneficial cellular processes and is also beneficial as a weight-loss strategy. - [Eat like an adult](https://thefitness.wiki/improving-your-diet/). Generally avoid processed foods, favor nuts and berries for snacks. **Eliminate sugars.** diff --git a/Open Data.md b/Open Data.md index bd76b72..7b82db6 100644 --- a/Open Data.md +++ b/Open Data.md @@ -2,7 +2,7 @@ [Open Data is a public good](https://en.wikipedia.org/wiki/Open_data#Open_Data_as_commons). As a result, individual [[incentives]] are not aligned with collective ones. -As an organization or research group, [spending time curating and maintaining datasets for other people to use doesn't make economic sense](https://en.wikipedia.org/wiki/Economics_of_open_data), unless you can profit from that. When a scientist publishes a paper, they care about the paper itself. They're incentivized to. The data is usually an afterthought. +As an [[Organizations|organization]] or research group, [spending time curating and maintaining datasets for other people to use doesn't make economic sense](https://en.wikipedia.org/wiki/Economics_of_open_data), unless you can profit from that. When a scientist publishes a paper, they care about the paper itself. They're incentivized to. The data is usually an afterthought. Combining data from different sources requires the user to reconcile the differences in [[Unified Schema Design|schemas]], formats, assumptions, and more. This [data wrangling is time-consuming, tedious and needs to be repeated by every user of the data](https://arxiv.org/pdf/2309.13054). @@ -10,23 +10,23 @@ The Open Data landscape has a few problems: - **Non-interoperability**. Data is isolated in multiple places and between different formats. - **Hard to Use**. Good datasets are hard to use as indexing is difficult and [many standards](https://xkcd.com/927/) compete. However, none of the indexers specify how the data is to be formatted, enforce any standardization, ... Users must still perform traditional forms of data merging, cleaning, and standardization. -- **No Collaboration**. No incentives exist for people to work on improving or curating datasets. +- **No Collaboration**. No [[Incentives|incentives]] exist for people to work on improving or curating datasets. - **No Versioning**. Datasets disappear or change without notice. It's hard to know what changed and when. Losing data doesn't just inconvenience a few researchers. It actively hinders scientific progress. -- **Economic Impact**. The inefficiency in data access and preparation represents a significant economic cost. E.g: thousands of data analysts spending 80% of their time preparing data for analysis, this represents billions in wasted economic opportunity. +- **Economic Impact**. The inefficiency in data access and preparation represents a significant economic cost. E.g: thousands of data analysts spending 80% of their [[Time|time]] preparing data for analysis, this represents billions in wasted economic opportunity. -[Open Data can help organizations, scientists, and governments make better decisions](https://twitter.com/patrickc/status/1256987283141492736). It drives innovation and decision-making across virtually every industry and sector. Data is one of the best ways to learn about the world and [[Coordination|coordinate]] people. Imagine if, every time you used a library, you had to find the original developer and hope they had a copy. It would be absurd. Yet that's essentially what we're asking scientists to do. [Science is missing a crucial [[Data Package Manager|packaging]]/publishing/sharing network](https://tom-nicholas.com/blog/2025/science-needs-a-social-network/). [Friction in data sharing hampers collaboration and limits informed decision-making](https://docs.google.com/document/d/1iTl7YWfTAzp8zNXRs01RAIWCP-pRJwQfDg8lsD0TDCM/edit?tab=t.0). +[Open Data can help organizations, scientists, and governments make better decisions](https://twitter.com/patrickc/status/1256987283141492736). It drives innovation and decision-making across virtually every industry and sector. Data is one of the best ways to learn about the world and [[Coordination|coordinate]] people. Imagine if, every time you used a library, you had to find the original developer and hope they had a copy. It would be absurd. Yet that's essentially what we're asking scientists to do. [Science is missing a crucial packaging/publishing/sharing network](https://tom-nicholas.com/blog/2025/science-needs-a-social-network/). [Friction in data sharing hampers collaboration and limits informed decision-making](https://docs.google.com/document/d/1iTl7YWfTAzp8zNXRs01RAIWCP-pRJwQfDg8lsD0TDCM/edit?tab=t.0). There are three big areas where people work on open data: at the government level covering thousands of datasets (CKAN, Socrata, …), at the scientific level (university level), and at the individual level where folks who are passionate about a topic publish a few datasets about it. This results in lots of datasets that are disconnected and still require you to scrape, clean, and join them from all the heterogeneous sources to answer interesting questions. [One of the big ways that data becomes useful is when it is tied to other data](https://x.com/auren/status/1139594779895844865). **Data is only as useful as the questions it can help answer**. Joining, linking, and [[Data IDE| graphing]] datasets together allows one to ask more and different kinds of questions. -Open protocols create open systems. Open code creates tools. **Open data creates open knowledge**. We need better tools, protocols, and mechanisms to improve the Open Data ecosystem. It should be easy to find, download, process, publish, and collaborate on open datasets. +Open protocols create open [[Systems|systems]]. Open code creates tools. **Open data creates open knowledge**. We need better tools, protocols, and mechanisms to improve the Open Data ecosystem. It should be easy to find, download, process, publish, and collaborate on open datasets. Iterative improvements over public datasets yield large amounts of value ([check how Dune did it with blockchain data](https://dune.com/blog/the-community-data-platform))¹. Access to data gives people the opportunity to create new business and make better decisions. Data is vital to understanding the world and improving public welfare. Metcalfe's Law applies to data too. The more connected a dataset is to other data elements, the more valuable it is. -In the blockchain example, data is Open, Verifiable, and Useful. And yet, the main provider of data is Dune, a company that captured most of the data layer. Users can run `cryo` but there are no incentives for them to share the data. There isn't a matchmaking market for data and people are forced to repeat the same work. +In the [[Blockchain|blockchain]] example, data is Open, Verifiable, and Useful. And yet, the main provider of data is Dune, a company that captured most of the data layer. Users can run `cryo` but there are no incentives for them to share the data. There isn't a matchmaking market for data and people are forced to repeat the same work. Open source code has made a huge impact in the world. Let's make Open Data do the same! Open data is, essentially, public infrastructure (similar to roads, bridges, or the internet). Let's make it possible for [anyone to fork and re-publish fixed, cleaned, reformatted datasets as easily as we do the same things with code](https://web.archive.org/web/20240719190206/https://juan.benet.ai/blog/2014-02-21-data-management-problems/). -This document is a collection of ideas and principles to make Open Data more accessible, maintainable, and useful. Also, recognizing that a lot of people are already working on this, there are some amazing datasets, tools, and organizations out there, and Open Data is an 80% people problem. This document is biased towards the technical side of things, as I think that's where I can contribute the most. I believe we can do much more with the available data. +This document is a collection of [[Ideas|ideas]] and principles to make Open Data more accessible, maintainable, and useful. Also, recognizing that a lot of people are already working on this, there are some amazing datasets, tools, and organizations out there, and Open Data is an 80% people problem. This document is biased towards the technical side of things, as I think that's where I can contribute the most. I believe we can do much more with the available data. ### Why Now? @@ -36,7 +36,7 @@ We have better and cheaper infrastructure. That includes things like faster stor During the last few years, a large number of new data and open source tools have emerged. There are new query engines (e.g: DuckDB, DataFusion, ...), execution frameworks (WASM), data standards (Arrow, Parquet, ...), and a growing set of open data marketplaces (Datahub, HuggingFace Datasets, Kaggle Datasets). "Small data" deserves more tooling and people working on it. There are many novel technologies too. -- [Differential Privacy](https://en.wikipedia.org/wiki/Differential_privacy) that allows releasing statistical information about datasets while protecting the privacy of individual data subjects. +- [Differential Privacy](https://en.wikipedia.org/wiki/Differential_privacy) that allows releasing statistical information about datasets while protecting the [[Privacy|privacy]] of individual data subjects. - Fully Homomorphic Encryption. - New [de-identification techniques](https://www.ipc.on.ca/sites/default/files/legacy/2016/08/Deidentification-Guidelines-for-Structured-Data.pdf). - Data watermarking, fingerprinting, and provenance tracking with blockchains. diff --git a/Open Source Data Projects.md b/Open Source Data Projects.md index f628424..2e75a23 100644 --- a/Open Source Data Projects.md +++ b/Open Source Data Projects.md @@ -1,6 +1,6 @@ # Open Source Data Projects -## Data Warehouses +## [[Databases|Data Warehouses]] - [ClickHouse](https://github.com/ClickHouse/ClickHouse) - [Seafowl](https://github.com/splitgraph/seafowl) @@ -20,13 +20,13 @@ - [Metabase](https://www.metabase.com/) - [Evidence](https://www.evidence.dev/) -## Visualizations +## [[Data Visualization|Visualizations]] - [Raw Graphs](https://app.rawgraphs.io/) - [Kepler](https://kepler.gl/) (maps) - [Observable Plot](https://observablehq.com/plot/) -## Data Pipelines +## [[Data Engineering|Data Pipelines]] - [Meltano](https://meltano.com) - [Airbyte](https://airbyte.io/) diff --git a/Openness.md b/Openness.md index b0b0556..945805f 100644 --- a/Openness.md +++ b/Openness.md @@ -2,7 +2,7 @@ - Embrace Weirdness. Don't be another human. Find your place, do what you love and put yourself outside your comfort zone. Unexpected and unconventional things make life more interesting. [If you fit in you disappear](https://twitter.com/tobi/status/1319017087948435458). - Celebrate and encourage quirky gifts, [[Habits]], behavior, and points of view. No one can compete with you on being you. -- Share your decisions, state, and ideas early so flaws arise earlier in the process. +- Share your decisions, state, and [[Ideas|ideas]] early so flaws arise earlier in the process. - Keep your [[Identity]] small. - Your job isn't to enforce your vision of the world upon everyone. Live your own life the way you want to live it and let other people live their own lives the way they want to live them. - [Don't yell at people who are trying to go off and do their own thing quietly with a group of voluntarily consenting friends](https://slatestarcodex.com/2014/06/07/archipelago-and-atomic-communitarianism). If people want to go do their own thing in a way that harms no one else, you _let_ them. @@ -12,11 +12,11 @@ - One of the hardest exercises is to receive [[feedback]]. Ask for it and be ready to change. Aim to improve once you've gathered feedback and reflected on it. - Criticizing your in-group takes courage but is possible. - Experience things again (books, movies, ...). They might be the same, but you change and are a different person now. The world changes, too. No one steps in the same river twice. -- Look for new experiences. It is the most important factor for [memorable experiences](https://web.archive.org/web/20240716123255/https://travelopment.com/how-to-create-memorable-experiences/). When all days follow the same routine, [there is no way to differentiate the memories](https://www.youtube.com/watch?v=zHL9GP_B30E). - - Monotony collapses time. Novelty unfolds it. You can exercise daily and eat healthily and live a long life, while experiencing a short one... Creating new memories stretches out psychological time, and lengthens our perception of our lives. +- Look for new experiences. It is the most important factor for [memorable experiences](https://web.archive.org/web/20240716123255/https://travelopment.com/how-to-create-memorable-experiences/). When all days follow the same [[Routine|routine]], [there is no way to differentiate the memories](https://www.youtube.com/watch?v=zHL9GP_B30E). + - Monotony collapses [[Time|time]]. Novelty unfolds it. You can exercise daily and eat healthily and live a long life, while experiencing a short one... Creating new memories stretches out psychological time, and lengthens our perception of our lives. - Keep high experimental velocity. Try more things quickly! - [When you learn an interesting idea, or read an article, it takes 0 effort to think through friends who might enjoy it, and pass it on](https://www.neelnanda.io/blog/mini-blog-post-10-seek-positive-externalities). This applies all the more so to bigger things - jobs worth applying to, other people they should talk to, etc. Receiving opportunities has (essentially) 0 downsides on their end. - Do stuff in public. Think in public. Write in public. Ship in public. [Doing things publicly attracts like-minded people and opportunities that you wouldn't otherwise be able to easily find](https://www.lesswrong.com/posts/dEAmXSyXnpFFc9wgd/21-on-21). -- You find what you like by trying it, not by thinking about it. +- You find what you like by trying it, not by [[Thinking|thinking]] about it. - Digital things don't have to be skeuomorphic. - When asking for something, go out of your way to make extremely clear (beforehand, not after a person has already answered) that [a no is perfectly fine](https://www.lesswrong.com/posts/GLpFovxZdwXYwmbkJ/failures-in-kindness) and really comes at no risk to the other person. diff --git a/Organizations.md b/Organizations.md index 9ea58d6..4562150 100644 --- a/Organizations.md +++ b/Organizations.md @@ -17,7 +17,7 @@ - All organizations have their own way of doing things ([[Culture]]) and their own particular views of how the world should work (vision). - **Vision** is the change in the world you want to see. - **Mission** defines the approach that you will execute to make your vision a reality. The **strategy** further refines the mission into concrete steps. - - There are two core processes that take place within any company: making decisions and doing work. + - There are two core processes that take place within any company: [[Making Decisions|making decisions]] and doing work. - Align [[incentives]]. Make the right thing to do the easiest thing to do. - The layout of the organization impacts how many hops the information has to make. - With each hop, the information loses some signal and increases the noise it carries. @@ -62,12 +62,12 @@ - Adoption of new [[processes]] should _always_ take the step-by-step iteration process into account. You should _never_ think of it as a system to be adopted wholesale; you should always think “hmm, this is a set of tools that seem to work for _some_ context; how do I know this works for mine?” And then you should break those processes down to atomized pieces, and apply them by running smaller, more iterative tests. Understanding the original context is usually key to understanding if the processes have a shot at working when applied to your company. - As in other complex [[systems]], we usually blame individuals instead of exploring the root causes. The problem is rarely someone being a villain. It's almost always the system. - [When facing organizational problems do not: ignore them, give up, switch to a top-down approach, involve more "lead people", rely on heroes or aim for perfection](https://komoroske.com/slime-mold/). -- Periodically observe pain points that cost productivity, and task people to reduce them, thereby producing more compound leverage across the whole organization. +- Periodically observe pain points that cost [[Productivity|productivity]], and task people to reduce them, thereby producing more compound leverage across the whole organization. - [In any organization, the people devoted to the benefit of the organization itself always gain control and those dedicated to the goals that the organization is supposed to accomplish have less and less influence, and sometimes are eliminated entirely](https://en.wikipedia.org/wiki/Jerry_Pournelle#Pournelle's_iron_law_of_bureaucracy). - Leaders in a healthy organization [[Communication|listen and communicate]]. - Organizations naturally move towards growth: more people, more systems, more conflicting worldviews. To facilitate action, communication systems _will_ be established. Make sure these are effective. - A company is a superorganism. -- [Teams usually don't stop to think about what things are working or not working, and why](https://cutlefish.substack.com/p/tbm-952-when-nothingeverything-actually). Make time for that. +- [Teams usually don't stop to think about what things are working or not working, and why](https://cutlefish.substack.com/p/tbm-952-when-nothingeverything-actually). Make [[Time|time]] for that. - By [swinging the pendulum](https://twitter.com/BrandonMChu/status/1502312472644100105) and changing focus periodically, you accept more extreme (and clear) outcomes in the short term, but in the long term arrive at the middle ground you aim for. - [Big organizations develop strategic inefficiency to carry on doing what they're doing](https://youtu.be/v1eWIshUzr8?t=1147). - You can't sell something like versioning or encryption as the key feature, you need to put out a comparable product and have these features be the thing that tips the scale in your favour. diff --git a/Pairwise Comparisons.md b/Pairwise Comparisons.md index b516647..fc3254c 100644 --- a/Pairwise Comparisons.md +++ b/Pairwise Comparisons.md @@ -7,13 +7,13 @@ Pairwise comparisons are any process of comparing entities in pairs to judge whi - Humans are better at relative judgments than absolute scoring. Scale-free comparisons reduce calibration headaches ([better judging systems](https://anishathalye.com/designing-a-better-judging-system/)). - Simple binary choices reduce cognitive load and make participation easier than intensity sliders. - Works across domains (psychometrics, recsys, sports, RLHF) where latent utility is hard to measure directly ([pairwise calibrated rewards](https://www.itai-shapira.com/pdfs/pairwise_calibrated_rewards_for_pluralistic_alignment.pdf)). -- Disagreement is signal. Diversity in raters surfaces semantics a single expert might miss ([CrowdTruth](https://github.com/CrowdTruth/CrowdTruth-core/blob/master/tutorial/Part%20I_%20CrowdTruth%20Tutorial.pdf)). +- [[Resolving Disagreement|Disagreement]] is signal. [[Plurality|Diversity]] in raters surfaces semantics a single expert might miss ([CrowdTruth](https://github.com/CrowdTruth/CrowdTruth-core/blob/master/tutorial/Part%20I_%20CrowdTruth%20Tutorial.pdf)). ## Collecting Good Data - Keep the UX fast and low-friction. Suggest options, keep context in the UI, and let people expand only if they want. - Avoid intensity questions. They are order-dependent and [require global knowledge](https://xkcd.com/883/). -- Use [active sampling](https://projecteuclid.org/journals/annals-of-statistics/volume-47/issue-6/Active-ranking-from-pairwise-comparisons-and-when-parametric-assumptions-do/10.1214/18-AOS1772.pdf)/dueling bandits to focus on informative pairs. Stop when marginal value drops. +- Use [active sampling](https://projecteuclid.org/journals/annals-of-statistics/volume-47/issue-6/Active-ranking-from-pairwise-comparisons-and-when-parametric-assumptions-do/10.1214/18-AOS1772.pdf)/dueling bandits to [[Focus|focus]] on informative pairs. Stop when marginal value drops. - With [efficiently sampled pairs](https://arxiv.org/abs/2302.13507) ([or approximate rankings](https://proceedings.mlr.press/v84/heckel18a.html)) far fewer comparisons are needed. - [Top-k tasks](https://proceedings.mlr.press/v84/heckel18a.html) can scale collection (pick best 3 of 6) while still convertible to pairwise data. - Expect [noisy raters](https://arxiv.org/abs/1612.04413). Filter or reweight after the fact using heuristics or gold questions instead of overfitting to ["experts" biases](https://link.springer.com/article/10.1007/s10618-024-01024-z). diff --git a/Parenting.md b/Parenting.md index a8c279f..0f2280a 100644 --- a/Parenting.md +++ b/Parenting.md @@ -17,16 +17,16 @@ - What works for one child may not work for another. It's important to be flexible and adjust your parenting style as needed to best support each child. - When tantrums occur, interrupt the flow of the moment. E.g: stop the car somewhere and say "[Do what you want but I'll restart the car once you are calm](https://news.ycombinator.com/item?id=31204423)". - Clearly communicated consequences ahead of time work much better than punishment. -- Raising responsible humans requires giving them responsibility, the opportunity for soft failures to learn from, and having those awkward long talks and figuring out how to set healthy boundaries. E.g: [how to deal with money](https://x.com/tayloramurphy/status/1824868300339249648). +- Raising responsible humans requires giving them responsibility, the opportunity for soft failures to [[Learning|learn]] from, and having those awkward long talks and figuring out how to set healthy boundaries. E.g: [how to deal with money](https://x.com/tayloramurphy/status/1824868300339249648). - Allow kids to have lots of small failures. Whenever you can, let them experience a small consequence that won't be too bad. - Allow them to experience the natural consequences of their actions as much as possible, both good and bad. Even if it has a negative consequence for them, as long as it's not too much. - Aim to be as predictable as possible. The brain is a prediction engine and hates surprises, so minimize the surprises they have to deal with. - For every strong opinion, there's an equally strong opposite opinion. Some basic strategies: - - Spend lots of (phone-free) time with them. + - Spend lots of (phone-free) [[Time|time]] with them. - Show them that the world is a fun and fascinating place. - - Don't interrupt them when they're focused or daydreaming. Help them learn to be entertained by their own minds. + - Don't interrupt them when they're [[Focus|focused]] or daydreaming. Help them learn to be entertained by their own minds. - Refrain from imposing lots of little rules, but where there are rules, be firm about them. - - Build problem-solving confidence, teaching them to become a person who says "I want to figure out the directions" instead of a person who says "let's just ask someone". + - Build [[Problem Solving|problem-solving]] confidence, teaching them to become a person who says "I want to figure out the directions" instead of a person who says "let's just ask someone". ## Resources diff --git a/Personal Handbooks.md b/Personal Handbooks.md index d95e57f..aea0855 100644 --- a/Personal Handbooks.md +++ b/Personal Handbooks.md @@ -1,9 +1,9 @@ # Personal Handbooks -Personal knowledge bases allow compound learning and make it easier to discover new abstractions or connections between concepts. Some interesting properties that a good personal knowledge base should have: +Personal knowledge bases allow compound [[Learning|learning]] and make it easier to discover new abstractions or connections between concepts. Some interesting properties that a good personal knowledge base should have: - Simple, open source and multi-platform. Works offline, uses plain text files, and has a backup mechanism similar to [[Git]]. -- Focus on capturing ideas and content as fast as possible. +- [[Focus]] on capturing [[Ideas|ideas]] and content as fast as possible. - Should be managed as a graph: - Nodes are notes or smaller graphs. [Notes should be atomic, concept-oriented and densely linked](https://notes.andymatuschak.org/z4SDCZQeRo4xFEQ8H4qrSqd68ucpgE6LU155C). - Bidirectional links connecting nodes. diff --git a/Piano.md b/Piano.md index 1af05cc..3af03b8 100644 --- a/Piano.md +++ b/Piano.md @@ -2,14 +2,14 @@ - [Getting Started as a Beginner](https://www.reddit.com/r/piano/wiki/faq#wiki_getting_started_as_a_beginner). - [Practicing for the absolute beginner](https://www.reddit.com/r/piano/comments/7porbg/practicing_for_the_absolute_beginner_where_and/) - Where and how to start playing the piano. -- [Synthesia](https://synthesiagame.com/) - A fun way to learn how to play the piano. +- [Synthesia](https://synthesiagame.com/) - A fun way to [[Learning|learn]] how to play the piano. - [How to Play Piano](https://www.youtube.com/watch?v=gnMDpBQ_bDQ&app=desktop) - Learn Pop Songs on the Piano in 5 Minutes. - [Beginner Piano](https://www.udemy.com/course/beginner-piano/) - Master keyboard fundamentals by learning popular tunes from "the best piano teacher in London". - [Getting Started with Music Theory](https://www.reddit.com/r/musictheory/wiki/faq/core/new_to_music_theory). More [lessons](https://www.musictheory.net/lessons) and a [great tool](https://play.google.com/store/apps/details?id=com.evilduck.musiciankit&hl=en). If you're in a hurry, check [How Music Works](https://www.lightnote.co/). - On the academic side, [Intro Theory by Dmitri Tymoczko](https://dmitri.mycpanel.princeton.edu/teaching.html). His course works on a much more fundamental level than any other music theory course. - [Study Music](https://github.com/vpavlenko/study-music) is a great GitHub repo with lots of music theory resources. -## YouTube Channels +## [[YouTube Channels]] - [Bill Hilton](https://www.youtube.com/user/billhiltonbiz) - [Nahre Sol](https://www.youtube.com/channel/UC8R8FRt1KcPiR-rtAflXmeg) diff --git a/Pixel Art.md b/Pixel Art.md index 7961d98..368e3f7 100644 --- a/Pixel Art.md +++ b/Pixel Art.md @@ -1,5 +1,7 @@ # Pixel Art +Pixel art is useful for [[Game Design|game design]] and small visual assets. + - [Tutorials](https://web.archive.org/web/20250730134304/https://saint11.art/blog/pixel-art-tutorials/) - Great collection of pixel art tutorials. ## Resources diff --git a/Planning.md b/Planning.md index 342aafc..0e1319c 100644 --- a/Planning.md +++ b/Planning.md @@ -9,7 +9,7 @@ Whether it comes naturally or is the result of a concerted effort and process, e - [[Coordination]]. - High shipping velocity. -The goal of planning is to get to a unified and realistic plan for what the system is collectively working towards over a given time horizon. It is not a one-time event, but a continuous process that happens at different levels of granularity. +The goal of planning is to get to a unified and realistic plan for what the system is collectively working towards over a given [[Time|time]] horizon. It is not a one-time event, but a continuous [[Processes|process]] that happens at different levels of granularity. To help with planning, you should have these things: diff --git a/Plurality.md b/Plurality.md index 1a81321..6797a82 100644 --- a/Plurality.md +++ b/Plurality.md @@ -21,6 +21,6 @@ - Plurality is compatible with: - Exponential technological growth, by designing property rights that force rising tides to lift all boats. - Valuing excellence and expertise, through mechanisms like prediction markets with per-person subsidies. - - Local experimentation in social media, blockchain ecosystems, and local government. -- [When different people disagree about how things should be done](https://vitalik.eth.limo/general/2025/12/17/societies.html), it's healthy to have a bias toward solutions where both versions exist at least somewhere, and people can freely choose between them. Peacefully making something small in a corner somewhere to see how it plays out is the most productive and least risky way for any actor to implement their vision. It provides rapid feedback about whether ideas actually work in reality. + - Local [[Experimentation|experimentation]] in social media, [[Blockchain|blockchain]] ecosystems, and local government. +- [When different people disagree about how things should be done](https://vitalik.eth.limo/general/2025/12/17/societies.html), it's healthy to have a bias toward solutions where both versions exist at least somewhere, and people can freely choose between them. Peacefully making something small in a corner somewhere to see how it plays out is the most productive and least risky way for any actor to implement their vision. It provides rapid [[Feedback|feedback]] about whether [[Ideas|ideas]] actually work in reality. - Ideas like this can enable open-source communities, collections of countries, and other groups that are not actors [to have higher levels of diffusion between each other](https://vitalik.eth.limo/general/2025/12/30/balance_of_power.html), allowing them to share more economies of scale and remain competitive with more internally organized centralized behemoths. diff --git a/Podcasts.md b/Podcasts.md index e862179..78cd63e 100644 --- a/Podcasts.md +++ b/Podcasts.md @@ -2,11 +2,11 @@ You can explore all the podcasts I'm subscribed to in [my Pocket Casts list](https://lists.pocketcasts.com/e0b1036d-ffe3-42af-ba4e-2c13a120a2b3). These are some of my favorite ones: -- [Lex Fridman Podcast](https://lexfridman.com/podcast/). Conversations about science, technology, history, philosophy and the nature of intelligence, consciousness, love, and power. +- [Lex Fridman Podcast](https://lexfridman.com/podcast/). Conversations about [[Science|science]], technology, history, philosophy and the nature of intelligence, consciousness, love, and power. - [Cortex](https://www.relay.fm/cortex). CGP Grey and Myke Hurley get together to discuss their working lives. - [Hardcore History](https://www.dancarlin.com/hardcore-history-series/). Explores topics throughout world history. - [Made You Think](https://madeyouthinkpodcast.com/). Episodes will explore books, essays, podcasts, and anything else that warrants further discussion, teaches something useful, or at the very least, exercises our brain muscles. -- [Mindscape](https://www.preposterousuniverse.com/podcast/). Conversations with the world's most interesting thinkers. Science, society, philosophy, culture, arts, and ideas. +- [Mindscape](https://www.preposterousuniverse.com/podcast/). Conversations with the world's most interesting thinkers. Science, society, philosophy, [[Culture|culture]], arts, and [[Ideas|ideas]]. - Spanish. - [Fall of Civilizations](https://fallofcivilizationspodcast.com/). A podcast that explores the collapse of different societies through history. - [Gabinete de Curiosidades](https://web.archive.org/web/20240528141801/https://www.gabinetepodcast.com/). diff --git a/Privacy.md b/Privacy.md index 10f1a85..6ff96aa 100644 --- a/Privacy.md +++ b/Privacy.md @@ -5,14 +5,14 @@ - [Doing privacy-first tackles multiple online harms by addressing corporate surveillance.](https://web.archive.org/web/20260224191356/https://www.eff.org/wp/privacy-first-better-way-address-online-harms) - [A society with no privacy is at a severe disadvantage in any resistance against the oppressive tendencies of the state and other large corporate actors.](https://www.reddit.com/r/privacy/comments/188v2ao/why_is_privacy_important_to_you/) - Think about how badly things can go if the wrong people end up in positions of power. - - What happens when someone is being stalked by an abusive ex-partner who works in law enforcement and has access to those databases? + - What happens when someone is being stalked by an abusive ex-partner who works in law enforcement and has access to those [[Databases|databases]]? - Data mining can be a weapon. - Privacy is the assurance that your data is only seen by the parties you intend to view it. - Anonymity is the ability to act without a persistent identifier. - Pseudonymity is a persistent identifier. - You shouldn't confuse privacy with secrecy. We know what happens in the bathroom, but you still close the door. -- Privacy is an important guarantor of decentralization: whoever has the information has the power, ergo we need to avoid centralized control over information. -- There are always certain facts about us, say, personal health information, or sexual behavior, that we wouldn't want the whole world to know. +- Privacy is an important guarantor of [[Decentralized Protocols|decentralization]]: whoever has the information has the power, ergo we need to avoid centralized control over information. +- There are always certain facts about us, say, personal [[Health|health]] information, or sexual behavior, that we wouldn't want the whole world to know. - [Less privacy => you give more data to advertisers => you get more personalized ads => ads are more effective on you => political ads change your opinions more => your political views are partly bought by money.](https://www.reddit.com/r/privacy/comments/188v2ao/why_is_privacy_important_to_you/). Ads work, on everyone, regardless of what you think. - You may be fine with being influenced by ads, but you are also imposing it on everyone you care about, without their consent. - Companies can see all your data, and companies are made of people, and people can be malicious. diff --git a/Problem Solving.md b/Problem Solving.md index 1cd46ef..c332b6b 100644 --- a/Problem Solving.md +++ b/Problem Solving.md @@ -20,7 +20,7 @@ - There are usually a number of possibilities for solving a problem. The first would be any solution that quickly gets the job done yet not actually permanent while the second would be a proper solution – which unfortunately takes longer and costs more. Know when to choose each option. [Nothing is more permanent than a temporary solution](https://thelightersideofwork.com/2021/04/nothing-is-more-permanent-than-a-temporary-solution/). - Solve problems by fixing the environment that allowed them to occur. - [Problem-solving can be understood as a search problem](https://rs.io/the-science-of-problem-solving/). You start in some state, there's a set of neighbor states you can move to, and a final state that you would like to end up in. - - Experiment more. Solve problems in different ways. + - [[Experimentation|Experiment]] more. Solve problems in different ways. - Focus on the changes you have to do between your state and the desired ones. - Taking breaks while working on problem-solving is called incubation. Incubation enhances problem-solving ability. - Problems are not disciplinary in nature but are holistic. @@ -34,7 +34,7 @@ - Break down large problems into smaller ones. [Chart a course and then create incentives for people to act/solve on them](https://youtu.be/B-dd2ZRlymo). [[Coordination]] is the key part of solving this kind of problem. - When we have a problem, our instinct is to add a new habit or purchase a fix. But sometimes, you can improve your life by taking things away ([Via Negativa](https://www.artofmanliness.com/articles/via-negativa-adding-to-your-life-by-subtracting/)). For example, the foods you avoid are more important than the foods you eat. - Your problems will tend to either be **adaptive** or **technical**. Adaptive problems require experimentation, novel strategies, or new ways of thinking and being; they're problems containing "unknown unknowns" and are often opaque in addition to being difficult. Technical problems may be equally difficult, but their difficulty lies in execution. Technical problems are those where the path to the solution is known or knowable and does not need to be discovered. -- [There's a kind of thinking you do without trying to](http://www.paulgraham.com/top.html). This type of thinking is not merely helpful in solving hard problems, but necessary. The tricky part is, you can only control it indirectly. Try to get yourself into situations where the most urgent problems are ones you want to think about. +- [There's a kind of thinking you do without trying to](http://www.paulgraham.com/top.html). This type of [[Thinking|thinking]] is not merely helpful in solving hard problems, but necessary. The tricky part is, you can only control it indirectly. Try to get yourself into situations where the most urgent problems are ones you want to think about. - If a problem has no solution, it may not be a problem, but a fact - not to be solved, but to be coped with over time. - Sometimes you can find the solution easier if you think about how not to solve the problem ([Inversion Principle](https://www.mymentalmodels.info/mms-inversion/)). - Keep the end goal in mind. [Don't Shave That Yak](https://seths.blog/2005/03/dont_shave_that/)! @@ -43,7 +43,7 @@ - To convince someone, show failed historical attempts and analyze them. - In complex problem spaces, [focus on direction](https://thecompendium.cards/c/everything/sort/stars/direction-not-solution) and not on the details. Don't focus too much on trying to find the best idea. The thing you have to prove is not that an idea is the best one, but that it's better than doing what you're currently doing. - [You cannot solve a people problem with a technical solution. Most technical problems are really people problems](https://blog.joeschrag.com/2023/11/most-technical-problems-are-really.html). Most people problems are [[Coordination]] and [[Communication]] problems. -- All technology produces second- and third-order effects beyond what was intended, and [[Feedback Loops]] often magnify them in complex systems. +- All technology produces second- and third-order effects beyond what was intended, and [[Feedback Loops]] often magnify them in complex [[Systems|systems]]. ## Five Whys diff --git a/Productivity.md b/Productivity.md index eeb82bc..153db4c 100644 --- a/Productivity.md +++ b/Productivity.md @@ -4,13 +4,13 @@ - Done is better than perfect. - Do the worst thing first. Give yourself a reward once you're done. - Adopt a fixed volume approach to productivity. - - Tough choices are inevitable; focus on making them consciously and well. + - Tough choices are inevitable; [[Focus|focus]] on making them consciously and well. - Create smaller chunks. This helps get into the flow (find the right balance between the challenge of the activity and your own skill level). - Avoid multitasking. Focus on one big project at a time, and see it to completion before moving on to the next. - Leverage the power of positive pressure by publicly committing to your goals. - [Improve your environment](https://nesslabs.com/neuroscience-of-procrastination). Design your workspace in a way that minimizes distractions, whether physical or digital. - **What is optimal for me won't be optimal for you**. You'll have to experiment to find out what works best for you. - [[Journaling|Keep a log]] of what happened each day. You can also add what you've learned! -- Create [[checklist|checklists]] for repetitive processes. For example, a [[checklist]] detailing all the tasks to do before ending the day. +- Create [[checklist|checklists]] for repetitive [[Processes|processes]]. For example, a [[checklist]] detailing all the tasks to do before ending the day. - "Where is the good knife?" If you're looking for your good X, it means you have bad Xs. Throw those out. - Your mind should be flexible, but your processes should be repeatable and predictable. diff --git a/Programming.md b/Programming.md index 52c4c44..66257d1 100644 --- a/Programming.md +++ b/Programming.md @@ -55,7 +55,7 @@ A programmer should know [lots](http://programmer.97things.oreilly.com/wiki/inde - **A programmer who can't reuse other programs is condemned to rewrite them**. - Use software leverage to your advantage. - Many programmers have only a superficial understanding of the importance of reusable code modules. - - [Code isn't just meant to be executed. Code is also a means of [[Communication]] across a team, a way to describe to others the solution to a problem](https://medium.com/s/story/notes-to-myself-on-software-engineering-c890f16f4e4d). Good writing skills often correlate with good thinking and programming skills. [Sharing knowledge through writing (blogs, talks, documentation, open source) clarifies your thinking and helps others](https://endler.dev/2025/best-programmers/). + - [Code isn't just meant to be executed. Code is also a means of communication across a team, a way to describe to others the solution to a problem](https://medium.com/s/story/notes-to-myself-on-software-engineering-c890f16f4e4d). Good [[Writing|writing]] skills often correlate with good thinking and programming skills. [Sharing knowledge through writing (blogs, talks, documentation, open source) clarifies your thinking and helps others](https://endler.dev/2025/best-programmers/). - **Silence is golden**. - A silent command is often more usable, providing the function asked for and nothing more. - **Think parallel**. diff --git a/Prompt Engineering.md b/Prompt Engineering.md index ee6a16a..6eabf39 100644 --- a/Prompt Engineering.md +++ b/Prompt Engineering.md @@ -1,15 +1,15 @@ # Prompt Engineering -- Designing prompts is an [[Agentic Engineering|iterative process]] that requires a lot of experimentation to get optimal results. +- Designing prompts is an [[Agentic Engineering|iterative process]] that requires a lot of [[Experimentation|experimentation]] to get optimal results. - Start with simple prompts and keep adding more elements and context as you aim for better results. - Learn about the [advanced prompting techniques](https://www.promptingguide.ai/techniques). - Be very specific about the instructions, task, and output format you want the [[Artificial Intelligence Models|model]] to perform. -- English is becoming the hottest new programming language. [Use it](https://addyo.substack.com/p/the-70-problem-hard-truths-about). +- English is becoming the hottest new [[Programming|programming]] language. [Use it](https://addyo.substack.com/p/the-70-problem-hard-truths-about). - [[Prompt Engineering|Prompts are code]]. Markdown and JSON files are state. - [Specs are core to programming with LLMs](https://www.dbreunig.com/2026/01/08/a-software-library-with-no-code.html). - Use comments to guide the model to do what you want. - Describe the problem very clearly and effectively. -- Make the model ask you more questions to refine the ideas. +- Make the model ask you more questions to refine the [[Ideas|ideas]]. - If you want to force some "reasoning", ask something like "[is that a good suggestion?](https://news.ycombinator.com/item?id=42894688)" or "propose a variety of suggestions for the problem at hand and their trade-offs". - Add relevant context to the prompt. Context can be external docs, a small pseudocode example, etc. Adding lots of context can confuse the model, so be careful! - ["Prompt engineering"](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) will have a large impact on the usefulness of an agent. diff --git a/Public Goods Funding.md b/Public Goods Funding.md index 093a3eb..9c84e31 100644 --- a/Public Goods Funding.md +++ b/Public Goods Funding.md @@ -8,7 +8,7 @@ Public goods are defined as goods that are both non-excludable (it's infeasible - Mathematical optimality matters less than perceived fairness and historical precedent. Ideal funding methods that don't work in practice are not ideal. - Many [[Mechanism Design|mechanisms]] which satisfy different constraints have already been discovered, and it seems unlikely that a different approach will radically change the landscape. Instead, the **bottleneck seems to be in popularizing and scaling existing mechanisms**. - Retrospective evaluation is often easier than prospective funding. [[Impact Evaluators]] and retroactive public goods funding reward **verifiable impact after the fact** instead of just predictions about future impact. -- Effective funding systems usually start small and local, with tight feedback loops and clear community ownership, and only then generalize once patterns are proven. +- Effective funding [[Systems|systems]] usually start small and local, with tight [[Feedback Loops|feedback loops]] and clear community ownership, and only then generalize once patterns are proven. - The funding **infrastructure itself is a public good**. Data, evaluation pipelines, and mechanisms should be open, composable, and forkable so communities can reuse and adapt them. ## Evaluating Allocations @@ -18,7 +18,7 @@ Public goods are defined as goods that are both non-excludable (it's infeasible - Useful portfolio-level questions: expected impact per dollar, additionality, concentration, representation, robustness to Sybils/collusion, and admin cost. - Replaying the same votes through different formulas is useful but limited: if another mechanism were binding, participants might vote, donate, collude, or apply differently. - Run shadow mechanisms, but reserve some budget for learning at the margin: randomized top-ups, just-above-vs-just-below-cutoff comparisons, or matched rejected applicants. -- A good program has to measure project outcomes over time and collect qualitative evidence to close the learning loop. It also helps organizers figure out if **the mechanism put more money into projects that later produced more impact** (can be computed with a regret metric). +- A good program has to measure project outcomes over [[Time|time]] and collect qualitative evidence to close the [[Learning|learning]] loop. It also helps organizers figure out if **the mechanism put more money into projects that later produced more impact** (can be computed with a regret [[Metrics|metric]]). ## Desirable Criteria @@ -28,14 +28,14 @@ Public goods are defined as goods that are both non-excludable (it's infeasible - **Budget Balance**. The mechanism generates sufficient revenue to cover its costs or payouts, without running a net deficit. - **Coalition-Proofness**. Preventing groups of participants from conspiring to manipulate the mechanism to their advantage. - **Provable Participation**. Even if spending should be kept private, users may want to prove their participation in a funding mechanism in order to boost their reputation or as part of an agreement. -- **Identity and Reputation**. To prevent Sybil attacks, some form of identity is needed. If reputation is important, a public identity is preferred. If anonymity is required, zero-knowledge proofs or re-randomizable encryption may be necessary. Reputation is an important incentive to fund public goods. Some form of reputation score or record of participation can be useful for repeated games. These scores can help identify bad actors or help communities coalesce around a particular funding venue. [Identity-free mechanisms can also be used](https://victorsintnicolaas.substack.com/p/funding-public-goods-in-identity-free-systems). +- **Identity and Reputation**. To prevent Sybil attacks, some form of [[Identity|identity]] is needed. If reputation is important, a public identity is preferred. If anonymity is required, zero-knowledge proofs or re-randomizable encryption may be necessary. Reputation is an important incentive to fund public goods. Some form of reputation score or record of participation can be useful for repeated games. These scores can help identify bad actors or help communities coalesce around a particular funding venue. [Identity-free mechanisms can also be used](https://victorsintnicolaas.substack.com/p/funding-public-goods-in-identity-free-systems). - **Verifiable Mechanisms**. Users may want certain guarantees about a mechanism before or after participation, especially if the mechanism being used is concealed. Ex-ante, they may want to upper-bound their amount of spending towards the good, ex-post, they may require proof that a sufficient number of individuals contributed. - **Anti-Collusion Infrastructure**. Like secure voting systems, there is a threat of buying votes in a funding mechanism. Collusion can be discouraged by making it impossible for users to prove how they reported their preferences. This infrastructure must be extended to prevent collusion between the 3rd party and the users. - **Predictable Schedules**. Participants need to know when they are getting funded. - **Simplicity and Legibility**. The simpler a mechanism (fewer parameters, clear rules, open-source and publicly verifiable execution), the less space there is for hidden privilege, corruption, and overfitting, and the easier it is for people to understand and engage with it. -- **Anti-Goodhart Resilience**. Any metric used for decisions will be gamed. Mechanisms should assume this, incorporate **feedback loops and error analysis**, and make it easy to update or combine metrics and evaluators when they drift from what really matters. +- **Anti-Goodhart Resilience**. Any metric used for decisions will be gamed. Mechanisms should assume this, incorporate **[[Feedback Loops|feedback loops]] and error analysis**, and make it easy to update or combine metrics and evaluators when they drift from what really matters. - **Plurality and Forkability**. No single mechanism can satisfy all desirable properties in all contexts. Systems should support **multiple evaluators and preference-aggregation methods**, and allow communities to fork and adapt criteria when they disagree. -- **Composable Data and Evaluation Layers**. Separate **data collection** (attestations about work, usage, dependencies, etc.) from **judgment** (how that data is weighted). Multiple evaluation "lenses" (models, juries, dashboards) should be able to operate on the same shared data structures (graphs, weight vectors). +- **Composable Data and Evaluation Layers**. Separate **data collection** (attestations about work, usage, dependencies, etc.) from **judgment** (how that data is weighted). Multiple evaluation "lenses" (models, juries, [[Dashboards|dashboards]]) should be able to operate on the same shared data structures (graphs, weight vectors). - **Exploration vs Exploitation**. Funding mechanisms are optimization processes that tend to exploit known winners. Some budget should be reserved for **exploration of uncertain, high-variance public goods**, not just those that already score well on existing metrics. - **Community Feedback and Local Control**. Mechanisms should include channels for participants to flag problems, suggest changes, and adjust evaluation criteria. Small, local experiments with clear consent and ownership are often the safest way to evolve funding systems. - **Model Marginal Returns**. A project may be excellent at $20k and wasteful at $200k. diff --git a/Public Speaking.md b/Public Speaking.md index d5c0ae9..48130c2 100644 --- a/Public Speaking.md +++ b/Public Speaking.md @@ -2,11 +2,11 @@ Some great advice for [public speaking presentations](https://web.archive.org/web/20230131031258/http://www.jilles.net/perma/2020/06/05/presentation-rules.html): -- Start the presentation with a set of questions that will be answered through the presentation. Start with an empowerment promise. Outline what the audience will know by the end of the talk. +- Start the presentation with a set of questions that will be answered through the presentation. Start with an empowerment promise. Outline what the audience will know by the end of the [[Talking|talk]]. - Find a way to connect with the audience right away. - Don't expose one bullet point of a list at a time. Your audience will ask to go back a slide or two, you end up halfway in a build, or worse, with an empty slide. None of this is needed. - Don't just read the words on a page or sound like you're delivering a memorized speech. -- [The best talks are stories. The best stories involve repetition.](https://speakerdeck.com/holman/the-talk-on-talks) Tell a personal story to make your ideas more sticky. +- [The best talks are stories. The best stories involve repetition.](https://speakerdeck.com/holman/the-talk-on-talks) Tell a personal story to make your [[Ideas|ideas]] more sticky. - Adjust your tempo of speaking to match the urgency of the story. - Anticipate the audience's skepticism and use it to your advantage. - Use fewer words. There is a place for details and context, but the most important is the actual message you want to get across. @@ -15,7 +15,7 @@ Some great advice for [public speaking presentations](https://web.archive.org/we - Structure. Stories can be told in many different ways. One way that works well in most situations is to divide the story arc up into Situation, Complication and Solution. Leaving one of these three components out will seriously hamper your ability to convince your audience, just like it is hard to ascend a staircase with stairs missing. - Number the slides. - Don't present any slide with a wall of text. -- Titles for the graphs should be the conclusion. +- Titles for the [[Data Visualization|graphs]] should be the conclusion. - If having issues keeping up with the story (e.g. you need long notes), use many slides to help you cover all the points and offer a summary at the end. - Write a great proposal. - Make your proposal easy to scan. diff --git a/Quotes.md b/Quotes.md index dbe3cee..96da396 100644 --- a/Quotes.md +++ b/Quotes.md @@ -18,9 +18,9 @@ A quote is a distilled piece of knowledge. These are some of my favorites. - Extraordinary claims require extraordinary evidence. Carl Sagan. - The rabbit runs faster than the fox, because the rabbit is running for his life while the fox is only running for his dinner. Richard Dawkins. - Where all think alike, no one thinks very much. Walter Lippmann. -- Every step and every movement of the multitude, even in what are termed enlightened ages, are made with equal blindness to the future; and nations stumble upon establishments, which [are indeed the result of human action, but not the execution of any human design](https://en.wikipedia.org/wiki/Spontaneous_order). Adam Ferguson. +- Every step and every movement of the multitude, even in what are termed enlightened ages, are made with equal blindness to the [[Future|future]]; and nations stumble upon establishments, which [are indeed the result of human action, but not the execution of any human design](https://en.wikipedia.org/wiki/Spontaneous_order). Adam Ferguson. - Comparison is the thief of joy. Theodore Roosevelt. -- I am writing a longer letter than usual because there is not enough time to write a short one. Blaise Pascal. +- I am [[Writing|writing]] a longer letter than usual because there is not enough [[Time|time]] to write a short one. Blaise Pascal. - When the facts change, I change my mind. What do you do, sir? John Maynard Keynes. - Reality is not optional. Thomas Sowell. - History doesn't repeat itself, but it does rhyme. Mark Twain. @@ -44,7 +44,7 @@ A quote is a distilled piece of knowledge. These are some of my favorites. - The past is a foreign country, they do things differently there. Leslie Poles Hartley. - Most advice is basically people mistaking correlation for causation. Leo Polovets. - More effort is wasted doing things that don't matter than is wasted doing things inefficiently. Elimination is the highest form of optimization. James Clear. -- Consensus is the process of abandoning all beliefs, principles, values, and policies in search of something in which no one believes, but to which no one objects; the process of avoiding the very issues that have to be solved, merely because you cannot get agreement on the way ahead. What great cause would have been fought and won under the banner: I stand for consensus?. Margaret Thatcher. +- Consensus is the process of abandoning all beliefs, principles, [[Values|values]], and policies in search of something in which no one believes, but to which no one objects; the process of avoiding the very issues that have to be solved, merely because you cannot get agreement on the way ahead. What great cause would have been fought and won under the banner: I stand for consensus?. Margaret Thatcher. - Y es que en el mundo traidor nada hay verdad ni mentira: todo es según el color del cristal con que se mira. Ramón de Campoamor. - You either move information from one place to another, or you move mass from one place to another. - The fewer the facts, the stronger the opinion. Arnold H. Glasow. diff --git a/README.md b/README.md index a0f96db..439f7d0 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ ![book](https://images.unsplash.com/photo-1483546363825-7ebf25fb7513?ixlib=rb-1.2.1&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=1740&q=80) -Personal bits of living knowledge, open for everyone. +Personal bits of living [[Learning|knowledge]], open for everyone. -- 🛠️ Explore [this Handbook on GitHub](https://github.com/davidgasquez/handbook). Send a [Pull Request](https://github.com/davidgasquez/handbook/pulls) or [file an issue](https://github.com/davidgasquez/handbook/issues) if you have any feedback! +- 🛠️ Explore [this Handbook on GitHub](https://github.com/davidgasquez/handbook). Send a [Pull Request](https://github.com/davidgasquez/handbook/pulls) or [file an issue](https://github.com/davidgasquez/handbook/issues) if you have any [[Feedback|feedback]]! - 🗺️ Explore any topic you find interesting on the left sidebar. - 📣 [Reach out](https://davidgasquez.com/) if you want to chat! diff --git a/Recipes.md b/Recipes.md index 62f0308..c0ed8b6 100644 --- a/Recipes.md +++ b/Recipes.md @@ -1,5 +1,7 @@ # Recipes +Useful [[Cooking|cooking]] references. + ## Vanilla Crepes ### Ingredients diff --git a/Relationships.md b/Relationships.md index 476b514..fa32d3b 100644 --- a/Relationships.md +++ b/Relationships.md @@ -4,10 +4,10 @@ - Build small groups. Small communities do more things than larger ones. - Do novel and exciting things with your friends often. - If you feel that something is not right, it probably is not right. Deal with it early. Problems are much quicker and easier to resolve when they're fresh and small than after you let them simmer into deep resentment. -- [Relationships are a coevolutionary loop](https://www.henrikkarlsson.xyz/p/making-a-home-together). Make it explicit by writing and sharing everything that goes wrong. This helps pinpoint the bottlenecks, the frictions. It also forces you to put words to your goals, values, and assumptions, opening those up for discussion and refinement. -- Acknowledge and apologize for any mistakes made. Learn and improve. +- [Relationships are a coevolutionary loop](https://www.henrikkarlsson.xyz/p/making-a-home-together). Make it explicit by [[Writing|writing]] and sharing everything that goes wrong. This helps pinpoint the bottlenecks, the frictions. It also forces you to put words to your [[Goals|goals]], [[Values|values]], and assumptions, opening those up for discussion and refinement. +- Acknowledge and apologize for any mistakes made. [[Learning|Learn]] and improve. - [Friends make life good](https://youtu.be/I9hJ_Rux9y0). They provide the scaffolding that makes it not just bearable but fun. They give us a sense of meaning and purpose and are a source of security, self-esteem and happiness. Almost nothing predicts how happy you will be as strongly as how connected you feel, and a lack of social connection is associated with a number of diseases and a shorter life. -- When presented with a challenging or boring moment with another person, deliberately adopt an attitude of curiosity in which your goal isn't to achieve any particular outcome or explain your position but to figure out who this human being is who we're with. +- When presented with a challenging or boring moment with another person, deliberately adopt an attitude of [[Curiosity|curiosity]] in which your goal isn't to achieve any particular outcome or explain your position but to figure out who this human being is who we're with. - Create many contexts. A friend who you see in only one context is likely to be a less close friend than someone who you see in many contexts, and connect with over many different things, rather than a single shared interest. - Be the first person to [reach out](https://www.neelnanda.io/blog/mini-blog-post-23-taking-social-initiative). Be the first person to organize events. Be the first person to say hi. Everyone is nice but you have to say hi first. -- Spend more time with fewer people. +- Spend more [[Time|time]] with fewer people. diff --git a/Remote Jobs.md b/Remote Jobs.md index 5c198a7..78dec2a 100644 --- a/Remote Jobs.md +++ b/Remote Jobs.md @@ -3,4 +3,4 @@ - [Remote Platforms List](https://docs.google.com/spreadsheets/d/1JfNAbUX_lN9K3MCNHO15GJtJ5qpk7H9Cl3xTBwv2FR8/htmlview) - Lots of websites with remote job offers. - [Awesome Remote Job](https://github.com/lukasz-madon/awesome-remote-job) - A curated list of awesome remote working resources. - [Remotive](https://remotive.io/) - Hand-screens live remote jobs from remote companies. -- [Outer Join](https://outerjoin.us/) - Remote jobs in data science. +- [Outer Join](https://outerjoin.us/) - Remote jobs in data [[Science|science]]. diff --git a/Resolving Disagreement.md b/Resolving Disagreement.md index ab6d4c8..03127c4 100644 --- a/Resolving Disagreement.md +++ b/Resolving Disagreement.md @@ -13,7 +13,7 @@ We should be arguing in a constructive fashion: treating arguments as an opportu - [[Curiosity]] and/or a desire to uncover truth. - As much data as possible to back up claims. [Cognitive biases](https://www.titlemax.com/discovery-center/lifestyle/50-cognitive-biases-to-be-aware-of-so-you-can-be-the-very-best-version-of-you/) are limits and mistakes in human judgment that prevent someone from acting rationally. They are present in every aspect of human life, and in tense situations like arguments, they tend to appear more often as emotions are heightened and the brain gets overloaded. - Agree on the terminology. Similar understanding of terms makes discussion more productive. -- Separate the topic from the community. E.g: cryptocurrencies have toxic communities but very interesting ideas. +- Separate the topic from the community. E.g: [[Cryptocurrencies|cryptocurrencies]] have toxic communities but very interesting [[Ideas|ideas]]. - If you debate it right, you'll end up respecting the other person. - [Sometimes, you'll end up with **high-level generators of disagreement**](https://slatestarcodex.com/2018/05/08/varieties-of-argumentative-experience/). They are what remain when everyone understands exactly what's being argued, and agrees on what all the evidence says, but has vague and hard-to-define reasons for disagreeing anyway. - [Make disagreement fun](https://twitter.com/waitbutwhy/status/1461620476363612169): @@ -29,7 +29,7 @@ We should be arguing in a constructive fashion: treating arguments as an opportu - Find the [[Double Crux]]. The point where you and your opponent's argument intersect. - Restate someone's point back to them to make sure you understood it correctly. This is called echoing and is more necessary for synchronous debates where information flows quickly. Not only does this make them feel heard, it also allows them the chance to clarify their beliefs. - Leave your partner a line of retreat. -- Instead of taking on a weaker version of your opponent's argument, help the entire debate out by thinking of the best and most charitable version of your opponent's argument, then repeat it back to them to see if it makes sense. +- Instead of taking on a weaker version of your opponent's argument, help the entire debate out by [[Thinking|thinking]] of the best and most charitable version of your opponent's argument, then repeat it back to them to see if it makes sense. Make sure you're arguing efficiently. We feel the most compulsion to distinguish ourselves from people who are most similar to us. We spend a lot of energy attacking other groups who agree with us 90% but we fight over the remaining 10%. diff --git a/Reverse ETL.md b/Reverse ETL.md index b2c4aca..d8eaccb 100644 --- a/Reverse ETL.md +++ b/Reverse ETL.md @@ -1,6 +1,6 @@ # Reverse ETL -- Data landscape moving like Unix tools. Specialized tools talking on the data warehouse. +- Data landscape moving like Unix tools. Specialized tools [[Talking|talking]] on the data warehouse. ## Why? @@ -13,8 +13,8 @@ - You get to use all the data you have, improving your [[Data Culture]]. - You can use the real source of truth for all the events and not rely on tracking only. - You can join sources like ChartMogul, Customer.io, etc. - - You can create more interesting events by enriching the events and user profiles with extra properties/traits (Trial Started with a conversion probability attached). Makes product analytics much more powerful. -- It is much easier to reuse the data available in the warehouse than it is to import the data in any new tool we use in the future. + - You can create more interesting events by enriching the events and user profiles with extra properties/traits (Trial Started with a conversion probability attached). Makes [[Product Analytics|product analytics]] much more powerful. +- It is much easier to reuse the data available in the warehouse than it is to import the data in any new tool we use in the [[Future|future]]. - You can be much more flexible with the tools we want to use because the data is shared and owned by us. - You avoid being locked into BI tools like Mixpanel since the logic will be stored in our warehouse. - As with any new tool, it gives more flexibility and power. diff --git a/Routine.md b/Routine.md index ab691cc..f15be34 100644 --- a/Routine.md +++ b/Routine.md @@ -2,7 +2,7 @@ A way to keep yourself on track with your [[habits]] is to have a routine. -1. **Rules**. Remind yourself of the current set of rules, constraints, various addictions, time-draining things. E.g. limit social media, do some stretching, ... +1. **Rules**. Remind yourself of the current set of rules, constraints, various addictions, [[Time|time]]-draining things. E.g. limit social media, do some stretching, ... 2. **Gratitude**. Visualize and [[Meditation|meditate]] on the idea that you might die today. 3. **Long-term goals and short-term [[goals]]**. Review big goals that you would like to achieve in the next years and the short-term goals that lead to them that you can act on. 4. **Visualize the day**. Visualize yourself going through the day's challenges and wins. @@ -21,11 +21,11 @@ Small lifestyle changes that can lead to large impact: - Hike often with friends. - [[Writing|Write]] more. - Consume content mindfully. -- Do experiments with life (A/B tests). -- Automate one thing. +- Do [[Experimentation|experiments]] with life (A/B tests). +- [[Automation|Automate]] one thing. - Chat with everyone. -- Actively reach out to friends and family. -- Take notes on everything you learn and share them. +- Actively reach out to [[Relationships|friends and family]]. +- Take notes on everything you [[Learning|learn]] and share them. - If it's under an hour, walk. - Have a day of the week where you list everything that went wrong during the weekend and figure out what needs to change. diff --git a/Science Fiction Reads.md b/Science Fiction Reads.md index 72607a7..0b9a23b 100644 --- a/Science Fiction Reads.md +++ b/Science Fiction Reads.md @@ -1,6 +1,6 @@ # Science Fiction Reads -Some great sci-fi reads I've enjoyed: +Some great [[Future|sci-fi]] reads I've enjoyed: - The Last Question. Asimov. - Project Hail Mary. Andy Weir. diff --git a/Science.md b/Science.md index f5dfd5e..34b121c 100644 --- a/Science.md +++ b/Science.md @@ -2,9 +2,9 @@ > The good thing about [science](https://youtu.be/3MRHcYtZjFY) is that it's true whether or not you believe in it. Neil deGrasse Tyson. -- Science is how humanity learns. +- Science is how humanity [[Learning|learns]]. - Science is the main way humanity extends what we know about the world beyond what we can sense directly. Technology is how we use that knowledge to make our lives better. -- Science itself isn't "true". It's a constantly refining process used to uncover truths based in material reality and that process is still full of mistakes. Do not treat science as a dogma, but rather as a refining process to uncover truths about reality. +- Science itself isn't "true". It's a constantly refining [[Processes|process]] used to uncover truths based in material reality and that process is still full of mistakes. Do not treat science as a dogma, but rather as a refining process to uncover truths about reality. - [Science is not clearly visible, like a comet bearing down on you](https://astralcodexten.substack.com/p/movie-review-dont-look-up). Science is like the Gnostic God. It exists, somewhere out there, perfect in itself. It is pure and right and beautiful. If you could hear it, it would certainly speak Truth. Yet here we are, in the stupid material universe, seeing through a glass darkly. Good sometimes looks like evil, evil often looks like good, and there's some jerk with the head of a lion and the body of a snake psyching us out at every turn. ## Metascience @@ -14,7 +14,7 @@ Metascience is the science of science itself - studying how scientific research - [Science's discovery ecosystem is currently in near stasis, with strong barriers preventing meaningful change](https://scienceplusplus.org/metascience/index.html). The social processes of science can be dramatically improved through systematic intervention. - Metascience operates as a three-part discipline combining imaginative design practice (envisioning radical alternatives), entrepreneurial discipline (building new [[Organizations|organizations]] and processes), and research field (rigorously evaluating effectiveness). - The **Metascience Learning Loop** provides a framework for systematic improvement: trial new social processes → evaluate effectiveness → scale successful approaches → amplify transformative changes. This mirrors [[Experimentation]] principles but applied to scientific institutions themselves. -- Key bottlenecks inhibiting scientific change include centralization of control, lack of clear change ownership, strong network effects, and absence of natural growth [[Feedback Loops]]. +- Key bottlenecks inhibiting scientific change include centralization of control, lack of clear change ownership, strong [[Network Effects|network effects]], and absence of natural growth [[Feedback Loops]]. - You can unlock science's potential by exploring radical institutional alternatives, developing generative heuristics to detect "intellectual dark matter," and taking entrepreneurial approaches to building new research [[Organizations]]. - Scientific [[Coordination]] problems are a specific manifestation of broader coordination challenges - addressing them requires both structural diversity and decentralized experimentation. diff --git a/Signaling.md b/Signaling.md index b6cd714..3159f07 100644 --- a/Signaling.md +++ b/Signaling.md @@ -2,7 +2,7 @@ - Signaling is [a method of conveying information among not-necessarily-trustworthy parties by performing an action which is more likely or less costly if the information is true than if it is not true](https://www.lesswrong.com/posts/KheBaeW8Pi7LwewoF/what-is-signaling-really). Because signals are often costly, they can sometimes lead to a depressing waste of resources, but in other cases they may be the only way to believably convey important information. - Most of our everyday actions can be traced back to some form of signaling or status seeking. - - A classic example of this would be a luxury watch: A Rolex isn't better at telling the time than a cheap Casio – but a Rolex signals something about its owner's economic power and thus their social standing. + - A classic example of this would be a luxury watch: A Rolex isn't better at telling the [[Time|time]] than a cheap Casio – but a Rolex signals something about its owner's economic power and thus their social standing. - When signaling, the more expensive and useless the item is, the more effective it is as a signal. Although eyeglasses are expensive, they're a poor way to signal wealth because they're very useful; a person might get them not because they are very rich but because they really need glasses. On the other hand, a large diamond is an excellent signal; no one needs a large diamond, so anybody who gets one anyway must have money to burn. - Our brains deliberately hide this fact from us and others (self-deception). Be [[Mindfulness|mindful]] of your actions. - [Signaling can be broken down into different components](https://julian.digital/2020/03/28/signaling-as-a-service/): diff --git a/Slack.md b/Slack.md index 95a89f9..13ebbd8 100644 --- a/Slack.md +++ b/Slack.md @@ -5,7 +5,7 @@ - If you want to try things, you have to have the slack to pick up errors. - Slack allows exploration. - [Slack prevents desperation and helps with long-term planning (by not having to put all the effort in the short term)](https://thezvi.wordpress.com/2017/09/30/slack/). -- When a part of a system lacks slack, it can't collaborate or help other parts. +- When a part of a [[Systems|system]] lacks slack, it can't collaborate or help other parts. - Adding slack to a system might make it more efficient. - Make sure that under normal conditions _you_ have Slack. Value it. Guard it. Spend it only when Worth It. If you lose it, fight to get it back. - [Slack creates _positive externalities_ for the group](https://www.lesswrong.com/posts/3qX2GipDuCq5jstMG/slack-has-positive-externalities-for-groups). On the other hand, it is not properly valued by the group. diff --git a/Sleep.md b/Sleep.md index c6b9252..c6a32ce 100644 --- a/Sleep.md +++ b/Sleep.md @@ -1,6 +1,6 @@ # Sleep -- Stick to a consistent routine. If possible, don't use an alarm to wake up. +- Stick to a consistent [[Routine|routine]]. If possible, don't use an alarm to wake up. - Don't use screens before sleep or right after waking up. - [Avoid sleep disruptors](https://alexvermeer.com/life-hacking/#sleep). Don't drink a lot of alcohol, don't eat large meals right before bed, don't have caffeine past mid-afternoon, don't watch super-intense movies right before bed, etc. - Spend the time to find the mattress, pillow, and sheets that work for you. diff --git a/Social Games.md b/Social Games.md index 5869d6c..658f283 100644 --- a/Social Games.md +++ b/Social Games.md @@ -5,7 +5,7 @@ - [Codenames](https://www.horsepaste.com/). Can also be played with [Dixit cards](https://meteuphoric.files.wordpress.com/2016/06/pic2664878_lg.jpg). - Wits and Wagers. - [Diplomacy](https://www.backstabbr.com/). -- [The Schelling Game](https://www.lesswrong.com/posts/kM3Xd2mJeWtsGkgSW/the-schelling-game-a-k-a-the-coordination-game). Discern Schelling points among the group. +- [The Schelling Game](https://www.lesswrong.com/posts/kM3Xd2mJeWtsGkgSW/the-schelling-game-a-k-a-the-coordination-game). Discern [[Coordination|Schelling points]] among the group. - [Aumann Agreement Game](https://docs.google.com/document/d/1gCKURs0Xdnb8PQS54rckS4CJUp8kCklKs2KKi7xDZdA/edit#). Also possible to [play online](https://web.archive.org/web/20241226132325/https://aumann.io/). - [Calibration Questions](https://web.archive.org/web/20230828073617/https://today.yougov.com/topics/politics/articles-reports/2022/03/15/americans-misestimate-small-subgroups-population). - [Black Stories](https://detective-stories.com/). @@ -29,7 +29,7 @@ - Guess Who. Someone asks a question like "Who is most likely to do X?" and everyone points to their guess at the same time. - Cheers to the Governor. As a group, you have to count to 21. The first rule is that 7 and 14 are swapped. Every time you succeed, you say cheers to the governor and add a new rule (e.g: 8 changes direction, 9 is silent, you have to say hello instead of 4, ...). - Celebrity cup. Add people to a cup and grab 2. You need to defend why that couple will work. -- Deduction Games like "Who Has the Hat". +- [[Thinking|Deduction]] Games like "Who Has the Hat". - Handmade [Chronology](https://boardgamegeek.com/boardgame/834/chronology). Write up events as a group, [then sort them out](https://boardgamegeek.com/video/47758/chronology/chronology-overview). - [Many other activities can be played as games](https://www.whataaabout.com/)! @@ -59,7 +59,7 @@ - Play some beats and everyone goes around freestyle rapping? - One Page RPGs. -## RPGs +## [[Dungeons and Dragons|RPGs]] - [Laser and Feelings](http://onesevendesign.com/lasers_and_feelings_rpg.pdf). Also [with hacks](https://writingalchemy.net/resources/lasers-and-feelings-hacks/)! - [Honey Heist](https://gshowitt.itch.io/honey-heist). @@ -68,7 +68,7 @@ - [Roll for Shoes](https://rollforshoes.com/). - [List of one-page RPGs](https://www.reddit.com/r/rpg/comments/1vie0k/what_are_your_favorite_onepage_rpgs/). -## Online Platforms +## [[Web Based Tools|Online Platforms]] - [Netgames](https://netgames.io/). Social games for your phone. - [Skribble.io](https://skribbl.io). Drawing game. @@ -78,7 +78,7 @@ - [Smart Cookie Trivia](https://web.archive.org/web/20251211062847/https://www.smart-cookie-trivia.com/). Free, fun, challenging, multiplayer quizzes. - [Escape Team](https://www.escape-team.com/). A digital-physical escape room game. -## Board Games +## [[Game Design|Board Games]] - [Scythe](https://boardgamegeek.com/boardgame/169786/scythe). - [Res Arcana](https://boardgamegeek.com/boardgame/262712/res-arcana). diff --git a/Spaced Repetition.md b/Spaced Repetition.md index 9b29b0a..cdfa53a 100644 --- a/Spaced Repetition.md +++ b/Spaced Repetition.md @@ -4,4 +4,4 @@ - Use one big deck. - [What makes Spaced Repetition better than conventional flashcards is that it manages the review schedule. If you can answer a question correctly, the time interval between reviews gradually expands](http://augmentingcognition.com/ltm.html). - Make cards following the [20 rules of formulating knowledge](https://www.supermemo.com/en/archives1990-2015/articles/20rules). -- If memorizing something will likely save you five minutes in the future, add it to your preferred Spaced Repetition Software. +- If memorizing something will likely save you five minutes in the [[Future|future]], add it to your preferred Spaced Repetition Software. diff --git a/Systems.md b/Systems.md index ca00a33..e3d175d 100644 --- a/Systems.md +++ b/Systems.md @@ -20,11 +20,11 @@ Keep in mind that intervening in a system requires some kind of theory, some kin [A complex system that works is invariably found to have evolved from a simple system that worked]() (more [elementary system functions](https://en.wikipedia.org/wiki/Systemantics#Elementary_systems_functions)). [Systems want to grow and grow](https://stephango.com/remove), but without pruning, they collapse. A good system is designed to be periodically cleared of cruft. It has a built-in counterbalance. -Complex systems usually have [attractor landscapes](https://ncase.me/attractors/) that can be used to change them. [The world is richer and more complicated than we give it credit for](https://slatestarcodex.com/2017/03/16/book-review-seeing-like-a-state/). To simplify it, we can focus on elements at different scales that have scale decoupling (quantum mechanics is decoupled from philosophy). +Complex systems usually have [attractor landscapes](https://ncase.me/attractors/) that can be used to change them. [The world is richer and more complicated than we give it credit for](https://slatestarcodex.com/2017/03/16/book-review-seeing-like-a-state/). To simplify it, we can [[Focus|focus]] on elements at different scales that have scale decoupling (quantum mechanics is decoupled from philosophy). Evolution is easier than revolution. A good approach to incrementally change a system (similar to [[Evolution|natural selection]]) is to: -1. Start by identifying the highest-leverage level to optimize at: Ask whether you're optimizing the machine or a cog within it. Complex systems might change in unexpected ways (butterfly effects). Minor differences in starting points make big differences on future states. +1. Start by identifying the highest-leverage level to optimize at: Ask whether you're optimizing the machine or a cog within it. Complex systems might change in unexpected ways (butterfly effects). Minor differences in starting points make big differences on [[Future|future]] states. 2. Begin optimizing the system by following the [Theory of Constraints](https://en.wikipedia.org/wiki/Theory_of_constraints): At any time, just one of a system's inputs is constraining its other inputs from achieving a greater total output. Make incremental changes. [Understand how the inputs affect the outputs of the system](https://faintsignal.org/pressure-to-meet-a-target-value-changes-the-system-or-the-data/#fn2). Alter the incentive landscape. [If you can make your system less miserable, make your system less miserable!](https://astralcodexten.substack.com/p/book-review-the-cult-of-smart) 3. Re-examine the system from the ground up. Get data. Take nothing but the proven, underlying principles as given. Work up from there to create something better. @@ -34,7 +34,7 @@ These are places within a complex system (a corporation, an economy, a living bo 2. The sizes of buffers and other stabilizing stocks, relative to their flows. 3. The structure of material stocks and flows (such as transport networks, population age structures). 4. The lengths of delays, relative to the rate of system change. -5. The strength of negative feedback loops, relative to the impacts they are trying to correct against. +5. The strength of negative [[Feedback|feedback]] loops, relative to the impacts they are trying to correct against. 6. The gain around driving positive feedback loops. 7. The structure of information flows (who does and does not have access to information). 8. The rules of the system (such as incentives, punishments, constraints). diff --git a/Technical Specifications.md b/Technical Specifications.md index f3e6416..5a3231d 100644 --- a/Technical Specifications.md +++ b/Technical Specifications.md @@ -11,9 +11,9 @@ A technical specification is a written design artifact you use to think before c - **Metadata**. Add the title, authors, reviewers, status, dates, and linked tickets so the document has clear accountability and revision context. - **Problem Statement and Context**. Explain the user or operational problem, why it matters, the relevant background, and the boundary of the system you are changing. -- **Goals, non-goals, and assumptions**. State what success includes, what is explicitly out of scope, and what conditions you rely on so reviewers can challenge scope early. +- **[[Goals]], non-goals, and assumptions**. State what success includes, what is explicitly out of scope, and what conditions you rely on so reviewers can challenge scope early. - **System overview and architecture**. Outline the main components, abstraction layers, dependencies, and how the pieces fit together before you dive into implementation detail. - **Domain model, interfaces, and invariants**. Define the important entities, data shapes, APIs or contracts, state machines, and normalization rules precisely enough that implementers make the same decisions. - **Execution, failure, and safety model**. Describe the main flows, failure modes, recovery behavior, trust boundaries, and operational safety rules for the system you are proposing. - **Test, rollout, and observability plan**. Say how you will validate the change, measure it in production, deploy it, and roll it back if needed. -- **Alternatives, risks, open questions, and definition of done**. Record trade-offs, unresolved decisions, success metrics, milestones, and the checklist that marks the work complete. +- **Alternatives, risks, [[Open Questions|open questions]], and definition of done**. Record trade-offs, unresolved decisions, success [[Metrics|metrics]], milestones, and the [[Checklist|checklist]] that marks the work complete. diff --git a/Unified Schema Design.md b/Unified Schema Design.md index 0a00b02..3049752 100644 --- a/Unified Schema Design.md +++ b/Unified Schema Design.md @@ -23,13 +23,13 @@ - `namespace`: Namespace of the attribute - `tags`: JSON with tags of the attribute? - `aggregation_function`: Aggregation function to use when aggregating the attribute - - `timeseries`: Timeseries are abstract measures (i.e. statistics, metrics) related to an entity and a date. Timeseries are temporal statistics or measures centered around an entity and timestamp. For example, GDP of Spain, population of Madrid, etc. Timeseries are abstract concepts (i.e. a measure) rather than a concrete thing. + - `timeseries`: Timeseries are abstract measures (i.e. statistics, [[Metrics|metrics]]) related to an entity and a date. Timeseries are temporal statistics or measures centered around an entity and timestamp. For example, GDP of Spain, population of Madrid, etc. Timeseries are abstract concepts (i.e. a measure) rather than a concrete thing. - Could be something like `weather_timeseries` to be able to join with the entities. - Columns: - `variable_id`: Unique identifier for the attribute - `geography_id`: Unique identifier for the geography - `date`: Date of the metric - `value`: Value of the metric - - `relationships`: Contains the relationships between entities. For example, Spain is composed of provinces, Madrid is a province, etc. + - `relationships`: Contains the [[Relationships|relationships]] between entities. For example, Spain is composed of provinces, Madrid is a province, etc. - Relationships can also be temporal – valid for an interval defined by specific start and end dates. - `characteristics`: Descriptors of an entity that are temporal. They have a start date and end date. diff --git a/Web Based Tools.md b/Web Based Tools.md index 32a1155..b611606 100644 --- a/Web Based Tools.md +++ b/Web Based Tools.md @@ -18,7 +18,7 @@ Collection of awesome web-based tools. - [SVG Crop](https://svgcrop.com/) - [SVGOMG](https://jakearchibald.github.io/svgomg/) -## Teamwork +## [[Teamwork]] - [HackMD](https://hackmd.io/) - [Matrix](https://matrix.org/) diff --git a/Writing Team Key Results.md b/Writing Team Key Results.md index af4b6cf..8754d39 100644 --- a/Writing Team Key Results.md +++ b/Writing Team Key Results.md @@ -13,8 +13,8 @@ Management techniques like OKRs and similar [have many issues](https://news.ycom - The key result does not presuppose the exact program of work to do (and in fact leaves room for creative solutions). - The key result states assumptions about why the team KR affects the top-level KR. - The key result target should be informed by analysis. -- Key Results are helpful if your team has a 12-18 month mission, a clear focus, and a specific lever or set of levers it is empowered to move. -- There is no replacement for good people, though. Project management is about organizing people. And people are hard to organize if they're not motivated, and adding a standardized, bureaucratized process won't increase motivation. +- Key Results are helpful if your team has a 12-18 month mission, a clear [[Focus|focus]], and a specific lever or set of levers it is empowered to move. +- There is no replacement for good people, though. Project management is about organizing people. And people are hard to organize if they're not motivated, and adding a standardized, bureaucratized [[Processes|process]] won't increase motivation. - OKRs are effectively a quarterly negotiation to get alignment across teams so that large, complex, projects get done on time. ## Process @@ -23,7 +23,7 @@ Management techniques like OKRs and similar [have many issues](https://news.ycom 2. State your assumptions: - Why does this team KR affect our company KR? - Why this target number? -3. Iterate and get feedback. +3. Iterate and get [[Feedback|feedback]]. 4. Lock it in. Three useful templates: diff --git a/Writing a Roadmap.md b/Writing a Roadmap.md index bf9800d..cee208b 100644 --- a/Writing a Roadmap.md +++ b/Writing a Roadmap.md @@ -1,10 +1,10 @@ # Writing a Roadmap -[Setting direction is one of the most important things you'll do when building a product and company. A clear direction aligns everyone to work toward the same goals](https://web.archive.org/web/20240423211458/https://linear.app/method/roadmap). It helps individuals make daily decisions, teams prioritize projects, and all members of your organization feel motivated toward a shared purpose. Without direction, it's harder to work together, know what to focus on, and make meaningful progress. +[Setting direction is one of the most important things you'll do when building a product and company. A clear direction aligns everyone to work toward the same goals](https://web.archive.org/web/20240423211458/https://linear.app/method/roadmap). It helps individuals make daily decisions, teams prioritize projects, and all members of your organization feel motivated toward a shared purpose. Without direction, it's harder to work together, know what to [[Focus|focus]] on, and make meaningful progress. Best practices to follow when building and managing a roadmap: -- Invest in planning +- Invest in [[Planning|planning]] - Break it down into milestones - Work on substantial projects - Prioritize for impact @@ -22,8 +22,8 @@ A roadmap needs to have different resolutions (zoom) tuned for different use cas - Purpose: What are we fighting for? - **Chunk progress into milestones**: Define the incremental targets to hit in order to accomplish the top-level [[Goals]]. What must be true to reach this milestone? On what timeline? With what dependencies? What does this unlock? - **Assign projects to milestones**: Fill in key projects that are necessary, nice to have, and out of scope to reach those milestones and bucket them accordingly. Why is each project important to achieve that milestone? Which are highest priority? Do they have any dependencies? Approximate how hard/big each project is. (This likely involves iterating on or shuffling the milestones themselves based on capabilities/effort.) -- **Iterate, iterate, iterate**: Review this draft with stakeholders to assess whether cross-cutting dependencies can be met, if you're prioritizing the right work, and which gaps or open questions you haven't thought of yet. -- **Finalize**: Get sign-off on this version of the roadmap for the upcoming time window. Present it to cross-functional partners and start using it as a communication device to align effort, priorities, and cross-team information sharing. +- **Iterate, iterate, iterate**: Review this draft with stakeholders to assess whether cross-cutting dependencies can be met, if you're prioritizing the right work, and which gaps or [[Open Questions|open questions]] you haven't thought of yet. +- **Finalize**: Get sign-off on this version of the roadmap for the upcoming [[Time|time]] window. Present it to cross-functional partners and start using it as a [[Communication|communication]] device to align effort, priorities, and cross-team information sharing. - **Expose the Roadmap**. It should be easy to view and collaborate on! ## Resources diff --git a/Writing.md b/Writing.md index a6be8d7..3a803b0 100644 --- a/Writing.md +++ b/Writing.md @@ -31,13 +31,13 @@ - Don't fully think through your ideas before writing. It's inefficient. The best way to think is by writing. It compels your brain to connect the dots. [Write whatever helps you think better](https://twitter.com/eugeneyan/status/1256828197410201601). - [Don't try to _persuade_ people that the idea is true/good. Instead, try to accurately _describe_ where the idea came from, the path which led _you_ to think it's true/plausible/worth a look. In the process, you'll probably convey your own actual level of uncertainty, which is exactly the right thing to do.](https://www.lesswrong.com/posts/Psr9tnQFuEXiuqGcR/how-to-write-quickly-while-maintaining-epistemic-rigor) - Be self-aware about your knowledge level on a topic, and say "I'm not sure…" when you are not sure about something. -- Separate the processes of creation from improvement. **You can't write and edit at the same time**. Write the first draft fast, then iterate on it editing things. Much of this editing will be cutting, and that makes simple writing even simpler. +- Separate the [[Processes|processes]] of creation from improvement. **You can't write and edit at the same [[Time|time]]**. Write the first draft fast, then iterate on it editing things. Much of this editing will be cutting, and that makes simple writing even simpler. - [Beware of "this"](https://www.lesswrong.com/posts/5e49dHLDJoDpeXGnh/editing-advice-for-lesswrong-users). Scan your words for words like "this" or "that", and when in doubt about clarity, replace them with whatever their intended antecedents are. - You can use tools like [Hemingway](http://www.hemingwayapp.com/) or [Ludwig](https://ludwig.guru/) to improve. - [The point of editing](https://evaparish.com/blog/how-i-edit) is to think about how you're using language and to make choices that suit the message you want to deliver, not to unquestioningly follow rules—mine or anyone else's. - When writing tutorials or guides, use the second-person and describe actions to a user. These types of content talk to people when humans can't. [Technical documentation follows the same rules as normal writing](https://developers.google.com/tech-writing/one). - [Make it fun](http://web.archive.org/web/20240307010045/https://davnicwil.com/tips-for-making-writing-more-fun/)! -- The skills you learn by writing transfer to speaking. Being good at speaking makes you more persuasive. +- The skills you [[Learning|learn]] by writing transfer to speaking. Being good at speaking makes you more persuasive. - If it's not written down, it's not. - Good nonfiction has three qualities: 1. It gets to the point. diff --git a/YouTube Channels.md b/YouTube Channels.md index 76f10c7..a75ade6 100644 --- a/YouTube Channels.md +++ b/YouTube Channels.md @@ -5,10 +5,10 @@ - [Kurzgesagt](https://www.youtube.com/user/Kurzgesagt). Videos explaining things with optimistic nihilism. - [exurb1a](https://www.youtube.com/channel/UCViwi-dXACoy9Pee73NvgJw). Explores our weird universe in a fun way. - [Tom Scott](https://www.youtube.com/user/enyay/videos?view=0&sort=p&flow=grid). Curiosities about many topics. -- [Veritasium](https://www.youtube.com/user/1veritasium). Videos about science, education, and anything else. +- [Veritasium](https://www.youtube.com/user/1veritasium). Videos about [[Science|science]], education, and anything else. - [Mark Rober](https://www.youtube.com/channel/UCY1kMZp36IQSyNx_9h4mpCg). Former NASA engineer. Current YouTuber and friend of science. - [3Blue1Brown](https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw). Maths and entertainment, depending on your disposition. -- [Primer](https://www.youtube.com/channel/UCKzJFdi57J53Vr_BkTfN3uQ). Communicating the deep ideas of academic subjects. +- [Primer](https://www.youtube.com/channel/UCKzJFdi57J53Vr_BkTfN3uQ). Communicating the deep [[Ideas|ideas]] of academic subjects. - [ColdFusion](https://www.youtube.com/channel/UC4QZ_LsYcvcq7qOsOhpAX4A). Stories about science, tech, and economics. - [Economics Explained](https://www.youtube.com/channel/UCZ4AMrDcNrfy3X6nsU8-rPg). Take a look at interesting countries, policies and decisions from the point of view of an economist. - [Simone Giertz](https://www.youtube.com/channel/UC3KEoMzNz8eYnwBC34RaKCQ). Maker/robotics enthusiast/non-engineer. Has become somewhat of an expert in shitty robots. @@ -18,5 +18,5 @@ - [LEMMiNO](https://www.youtube.com/c/LEMMiNO). Documentaries and list videos. - [Philosophy Tube](https://www.youtube.com/c/thephilosophytube/). Giving away a philosophy degree on YouTube. - [ContraPoints](https://www.youtube.com/c/ContraPoints). Interesting essays on philosophy topics. -- [Adam Ragusea](https://www.youtube.com/user/aragusea). Home cooking videos marinated with some science. +- [Adam Ragusea](https://www.youtube.com/user/aragusea). Home [[Cooking|cooking]] videos marinated with some science. - [You Suck At Cooking](https://www.youtube.com/c/yousuckatcooking). Fun cooking videos. -- 2.51.2