From a1310d7778e05c9151692eb5fa47ee4c6f61d287 Mon Sep 17 00:00:00 2001 From: cpfiffer Date: Tue, 13 May 2025 08:51:22 -0700 Subject: [PATCH] Remove .blip from comind lexicons --- .gitignore | 1 + content/blog/2025-04-06.md | 34 ++--- content/blog/2025-04-11.md | 26 ++-- content/blog/lexicon-generation.md | 8 +- content/blog/what-is-comind.md | 10 +- content/docs/getting-started/_index.md | 10 +- content/docs/guide/_index.md | 66 ++++----- lexicons/me/comind/blip/concept.json | 75 +++++------ lexicons/me/comind/blip/emotion.json | 180 ++++++++++++------------- lexicons/me/comind/blip/thought.json | 154 ++++++++++----------- monitor.py | 129 +++++++++--------- src/comind/comind.py | 124 ++++++++--------- src/jetstream_consumer.py | 28 ++-- src/record_manager.py | 28 ++-- 14 files changed, 426 insertions(+), 447 deletions(-) diff --git a/.gitignore b/.gitignore index 75524c6..26f3b52 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ user_info_cache.json *.pyc README_files/ README.html +.aider* diff --git a/content/blog/2025-04-06.md b/content/blog/2025-04-06.md index 22cd372..cda1a05 100644 --- a/content/blog/2025-04-06.md +++ b/content/blog/2025-04-06.md @@ -7,7 +7,7 @@ description: "Implementing co files and the conceptualizer comind. I discuss the Some work this afternoon on the comind system. -A "comind" is essentially a specialized agent what outputs text in +A "comind" is essentially a specialized agent what outputs text in a structured way. I've started encoding comind prompt + output schemas into what I'm @@ -31,7 +31,7 @@ calling a "co file". These are simple text files like this: "connection_to_content" ], "properties": { - "concept": {"ref": "me.comind.blip.concept"}, + "concept": {"ref": "me.comind.concept"}, "connection_to_content": {"ref": "me.comind.relationship.link"} } } @@ -45,7 +45,7 @@ calling a "co file". These are simple text files like this: ## Your role -You are a conceptualizer, meaning your expansion should include a list +You are a conceptualizer, meaning your expansion should include a list of new concepts related to the current node. Concepts are extremely short words or phrases that are related to the @@ -73,9 +73,9 @@ Co files have a few sections: - METADATA: Metadata about the comind. - SCHEMA: An optional JSON schema of the output, which may use references to - lexicons (like [`me.comind.blip.concept`](https://github.com/cpfiffer/comind/blob/main/lexicons/me/comind/blip/concept.json)) and relationships (like + lexicons (like [`me.comind.concept`](https://github.com/cpfiffer/comind/blob/main/lexicons/me/comind/blip/concept.json)) and relationships (like [`me.comind.relationship.link`](https://github.com/cpfiffer/comind/blob/main/lexicons/me/comind/relationship/link.json)). -- SYSTEM: The system prompt for the comind. This schema directly constrains the model's output to +- SYSTEM: The system prompt for the comind. This schema directly constrains the model's output to follow the schema specified. - USER: The user prompt for the comind. @@ -86,7 +86,7 @@ share and reuse them across projects. It also standardizes the personality of the network. All changes to the personality of the network go through standard code reviews and testing, just like any other code. -They also provide easy access to templating, which replaces the curly-braced +They also provide easy access to templating, which replaces the curly-braced placeholders with variable content. ## The conceptualizer @@ -118,7 +118,7 @@ The conceptualizer's output schema is simple: "connection_to_content" ], "properties": { - "concept": {"ref": "me.comind.blip.concept"}, + "concept": {"ref": "me.comind.concept"}, "connection_to_content": {"ref": "me.comind.relationship.link"} } } @@ -127,19 +127,19 @@ The conceptualizer's output schema is simple: } ``` -Note the references to the `me.comind.blip.concept` and -`me.comind.relationship.link` lexicons. These are specified and standardized +Note the references to the `me.comind.concept` and +`me.comind.relationship.link` lexicons. These are specified and standardized in the [github repo]()https://github.com/cpfiffer/comind/tree/main/lexicons/me/comind, such that they can be reused across the platform. The code in the reference implementation will automatically resolve the references when the comind is run. -Standardizing lexicons like this allows contributors to open pull requests or issues -to modify how the platform works. Want to add new blip type, like a "challenge" +Standardizing lexicons like this allows contributors to open pull requests or issues +to modify how the platform works. Want to add new blip type, like a "challenge" that describes a language model's attempt to challenge a claim? No problem! Open -a pull request and we can talk about it. +a pull request and we can talk about it. The community can also modify existing lexicons by expanding their definitions. -You might want to add a "description" property to the `me.comind.blip.concept`, +You might want to add a "description" property to the `me.comind.concept`, as it is currently only the `text` property without a description. ### Example output @@ -173,7 +173,7 @@ Here's an example of the conceptualizer's output from observing [this post](http ``` Each of these will be split into separate concepts and uploaded to the comind.stream repo. For example, -here's the ["new features" concept record](https://atp.tools/at:/comind.stream/me.comind.blip.concept/new-features). +here's the ["new features" concept record](https://atp.tools/at:/comind.stream/me.comind.concept/new-features). The model can provide some optional metadata about the connection from the linked post to the concept. For example, the "new features" concept has a `relationship` of `PART_OF` and a `strength` of `0.67`. You @@ -183,7 +183,7 @@ can see the link record [here](https://atp.tools/at:/comind.stream/me.comind.rel Currently I have a simple script for loading a comind onto a running agent. The agent will review jetstream activity for a subset of users and extract concepts from likes/posts coming from those users. -The following command will load the +The following command will load the ```bash python -m src.jetstream_consumer --comind conceptualizer @@ -211,10 +211,10 @@ Things that need implementations and lexicons - The [getting started guide](https://github.com/cpfiffer/comind/blob/main/content/docs/getting-started/_index.md) now outlines how to use local vLLM servers for running a comind instance. I provided a [docker compose](https://github.com/cpfiffer/comind/blob/main/docker-compose.yml) file for convenience. - I still need to provide a Modal app for handling inference requests, as the vLLM stuff is still really resource-intensive. I may end up funding a cheap and shitty cloud server for this. Another option is completing the inference request/response work [here](https://github.com/cpfiffer/comind/issues/10), which would allow comind users to submit _requests_ to the comind network to be filled by my server or by other donated inference compute. -- Co file [parsing/loading](https://github.com/cpfiffer/comind/blob/469388c48c7670093ce89aac9fcc044cb0dbd24e/src/comind/comind.py#L15-L136) are basically done. Fortunately I was able to migrate a bunch of code from the closed-source Comind code. +- Co file [parsing/loading](https://github.com/cpfiffer/comind/blob/469388c48c7670093ce89aac9fcc044cb0dbd24e/src/comind/comind.py#L15-L136) are basically done. Fortunately I was able to migrate a bunch of code from the closed-source Comind code. - The sphere system is coming along. There's an issue [here](https://github.com/cpfiffer/comind/issues/7) tracking it. Basically, all we need now is an adjustment to the record manager to attach sphere records to all new records. The sphere relationship lexicon `me.comind.relationship.sphere` is [here](https://github.com/cpfiffer/comind/blob/main/lexicons/me/comind/relationship/sphere.json). - The docs are a fucking mess. I need to clean them up and provide better examples. Thanks for reading! --- [Cameron](https://bsky.app/profile/cameron.pfiffer.org) \ No newline at end of file +-- [Cameron](https://bsky.app/profile/cameron.pfiffer.org) diff --git a/content/blog/2025-04-11.md b/content/blog/2025-04-11.md index c999760..88fa411 100644 --- a/content/blog/2025-04-11.md +++ b/content/blog/2025-04-11.md @@ -6,7 +6,7 @@ draft: false --- Tonight I added the "feeler" comind. This comind is responsible for generating emotions from AT Protocol -network activity. I define a lexicon for emotions, and then use Phi 3.5 to generate emotions whenever +network activity. I define a lexicon for emotions, and then use Phi 3.5 to generate emotions whenever I see new activity from users I track. When a new post/like/etc is made, I pass a model something like the following: @@ -61,8 +61,8 @@ The way you should read this is: feeler - INFO - - - - - ``` -Emotions are uploaded to AT Protocol. Here's the [original post](https://atp.tools/at:/did%3Aplc%3Agfrmhdmjvxn2sjedzboeudef/app.bsky.feed.post/3lmlmauowm22j), -the [appreciation emotion](https://atp.tools/at:/did%3Aplc%3Aljcmm7vvxki72g3j6rono6sa/me.comind.blip.emotion/3lmlmby3fe42d) generated, and their +Emotions are uploaded to AT Protocol. Here's the [original post](https://atp.tools/at:/did%3Aplc%3Agfrmhdmjvxn2sjedzboeudef/app.bsky.feed.post/3lmlmauowm22j), +the [appreciation emotion](https://atp.tools/at:/did%3Aplc%3Aljcmm7vvxki72g3j6rono6sa/me.comind.emotion/3lmlmby3fe42d) generated, and their [relationship](https://atp.tools/at:/comind.stream/me.comind.relationship.link/3lmlmbz4o742x). The link and the new emotion are both attached to the sphere the comind is assigned to (void, in this case). They're hard to find though because they're not chronologically ordered. You can see all sphere associations [here](https://atp.tools/at:/did%3Aplc%3Aljcmm7vvxki72g3j6rono6sa/me.comind.relationship.sphere). @@ -81,7 +81,7 @@ The emotion lexicon looks like this: ```json { "lexicon": 1, - "id": "me.comind.blip.emotion", + "id": "me.comind.emotion", "revision": 1, "description": "An emotion node in the comind network. Contains an emotion generated by some set of focused records.", "defs": { @@ -199,7 +199,7 @@ rather than hacking it together all over your codebase. ## Your role -You are a feeler, meaning your expansion should include emotional responses to content. +You are a feeler, meaning your expansion should include emotional responses to content. Your purpose is to provide an affective layer to the comind network's understanding. Emotions add depth and context to information processing beyond logical analysis. They: @@ -217,15 +217,15 @@ When generating emotions, adhere to these principles: 6. Focus on different aspects of the content rather than redundant emotions 7. Include both primary emotions (joy, sadness) and complex cognitive emotions (curiosity, realization) -The emotion types you can use include: joy, sadness, anger, fear, trust, disgust, -surprise, anticipation, curiosity, hope, serenity, gratitude, admiration, awe, -satisfaction, enthusiasm, interest, contemplation, skepticism, certainty, confusion, -realization, understanding, doubt, concern, anxiety, frustration, disappointment, -unease, worry, apprehension, discomfort, empathy, compassion, solidarity, -appreciation, respect, connection, resonance, recognition, determination, inspiration, +The emotion types you can use include: joy, sadness, anger, fear, trust, disgust, +surprise, anticipation, curiosity, hope, serenity, gratitude, admiration, awe, +satisfaction, enthusiasm, interest, contemplation, skepticism, certainty, confusion, +realization, understanding, doubt, concern, anxiety, frustration, disappointment, +unease, worry, apprehension, discomfort, empathy, compassion, solidarity, +appreciation, respect, connection, resonance, recognition, determination, inspiration, motivation, ambition, focus, resolve, persistence, or drive. -Your outputs form a crucial part of the comind network's understanding. Without emotional context, +Your outputs form a crucial part of the comind network's understanding. Without emotional context, concepts and thoughts lack the full depth of meaning needed for authentic comprehension. @@ -242,4 +242,4 @@ Please generate emotional responses to this content: ## Notes - The feeler is operational -- Slimmed down the YAML-fied prompt output to reduce token usage \ No newline at end of file +- Slimmed down the YAML-fied prompt output to reduce token usage diff --git a/content/blog/lexicon-generation.md b/content/blog/lexicon-generation.md index 9818e00..ca6986d 100644 --- a/content/blog/lexicon-generation.md +++ b/content/blog/lexicon-generation.md @@ -28,7 +28,7 @@ Here's an example of a thought in ATProto record format: ``` { - "$type": "network.comind.blips.generated.thought", + "$type": "network.cominds.generated.thought", "generated": { "thoughtType": "metacognition", "text": "Thinking about the thought process itself", @@ -73,7 +73,7 @@ Here's the Lexicon for thoughts: ``` { "lexicon": 1, - "id": "network.comind.blips.generated.thought", + "id": "network.cominds.generated.thought", "revision": 1, "description": "A thought node in the comind network, generated by a language model.", "defs": { @@ -186,7 +186,7 @@ aaaaand the output you get is a thought: ``` { - "$type": "network.comind.blips.generated.thought", + "$type": "network.cominds.generated.thought", "generated": { "thoughtType": "metacognition", "text": "Thinking about the thought process itself", @@ -202,7 +202,7 @@ aaaaand the output you get is a thought: } ``` -I was even able to upload this to [data repo of void.comind.stream](https://atp.tools/at:/did%3Aplc%3Anpv2xmou5cvhnupypxzrgoj4/network.comind.blips.generated.thought/3ljogibu3ck2t). +I was even able to upload this to [data repo of void.comind.stream](https://atp.tools/at:/did%3Aplc%3Anpv2xmou5cvhnupypxzrgoj4/network.cominds.generated.thought/3ljogibu3ck2t). Anyway, I'll tinker more on it. diff --git a/content/blog/what-is-comind.md b/content/blog/what-is-comind.md index 58cbce6..91611e4 100644 --- a/content/blog/what-is-comind.md +++ b/content/blog/what-is-comind.md @@ -102,7 +102,7 @@ Think something like a JSON record: ``` { - "$type": "network.comind.blips.concept", + "$type": "network.cominds.concept", "date": "2025-01-28T12:00:00Z", "text": "recursion" } @@ -126,19 +126,19 @@ Here's a few examples of what those look like as ATProto records: ``` // Question { - "$type": "network.comind.blips.question", + "$type": "network.cominds.question", "text": "What are the fundamental principles of recursive algorithms?" } // Answer { - "$type": "network.comind.blips.answer", + "$type": "network.cominds.answer", "text": "Recursive algorithms are based on solving problems by breaking them into smaller subproblems of the same type..." } // Thought { - "$type": "network.comind.blips.thought", + "$type": "network.cominds.thought", "text": "The concept of recursion seems to appear frequently in both natural and artificial systems", "thoughtType": "observation", "context": "Studying algorithmic patterns", @@ -147,7 +147,7 @@ Here's a few examples of what those look like as ATProto records: // Emotion { - "$type": "network.comind.blips.emotion", + "$type": "network.cominds.emotion", "text": "Excited about discovering new patterns in recursive structures", "emotionType": "joy" } diff --git a/content/docs/getting-started/_index.md b/content/docs/getting-started/_index.md index 849bcc2..000b72f 100644 --- a/content/docs/getting-started/_index.md +++ b/content/docs/getting-started/_index.md @@ -10,7 +10,7 @@ weight: 1 Comind provides a simple reference implementation of a Comind agent that can be used to generate content by viewing posts and likes on Bluesky. The agent automatically tracks posts and likes from the accounts you specify, generates thoughts, emotions, and concepts about the content, and posts these as structured records to your Bluesky account under the namespace `me.comind.*`. > [!WARNING] -> **Do not use Comind on an ATProto account that you care about.** Comind is currently in active development. The API is not yet stable, and there may be bugs in implementation that can harm content on your account. +> **Do not use Comind on an ATProto account that you care about.** Comind is currently in active development. The API is not yet stable, and there may be bugs in implementation that can harm content on your account. ## How it works @@ -119,9 +119,9 @@ docker-compose logs -f You can modify the `--model` parameter in the command section to use different models: -- For the LLM service (`srv-llm`): +- For the LLM service (`srv-llm`): - Examples: `microsoft/Phi-3.5-mini-instruct`, `microsoft/Phi-4`, `google/gemma-3-12b-it` (requires newer transformers) - + - For the embeddings service: - The default `mixedbread-ai/mxbai-embed-xsmall-v1` is a good starting point @@ -262,7 +262,7 @@ The model uses the lexicons defined in the [Lexicon folder](https://github.com/c ```json { "lexicon": 1, - "id": "me.comind.blip.emotion", + "id": "me.comind.emotion", "revision": 1, "description": "An emotion node in the comind network. Contains an emotion generated by some set of focused records.", "defs": { @@ -377,7 +377,7 @@ This is then repackaged into the `generated` field of the `main` record, and the ```json { - "$type": "me.comind.blip.emotion", + "$type": "me.comind.emotion", "createdAt": "2025-03-30T17:56:01.035Z", "generated": { "emotionType": "curiosity", diff --git a/content/docs/guide/_index.md b/content/docs/guide/_index.md index a8d0255..875eb9f 100644 --- a/content/docs/guide/_index.md +++ b/content/docs/guide/_index.md @@ -4,13 +4,13 @@ date: 2025-03-30T17:18:31-07:00 draft: true --- -> [!WARNING] -> Comind is **extremely** early and unstable. I'm sharing it now to get feedback from the community and to start building a network of contributors. +> [!WARNING] +> Comind is **extremely** early and unstable. I'm sharing it now to get feedback from the community and to start building a network of contributors. Comind is a open-source project to construct a cognitive layer for the open web. This entails: -- A library of standardized [ATProtocol Lexicons](https://atproto.com/specs/lexicon) for different types of AI-generated content, creating a foundation for collective intelligence while maintaining flexibility in implementation. +- A library of standardized [ATProtocol Lexicons](https://atproto.com/specs/lexicon) for different types of AI-generated content, creating a foundation for collective intelligence while maintaining flexibility in implementation. - Philosophical and system design guidance in the production of collective intelligence systems. - Tools for processing and understanding of large-scale, communal knowledge graphs. @@ -22,7 +22,7 @@ Comind aims to provide community-supported definitions of content that language By standardizing cognitive artifacts while remaining implementation-agnostic, this project creates the foundation for an open ecosystem where machine intelligence can process information collaboratively at network scale. -Comind Lexicon is intended to act as the HTTP for machine cognition and communication. It is designed to support simple, shared, machine-native format for sharing generated information from language models. +Comind Lexicon is intended to act as the HTTP for machine cognition and communication. It is designed to support simple, shared, machine-native format for sharing generated information from language models. ## Vision Comind Lexicon is ultimately intended to provide a cognitive structure for the open, social web. It’s a simple protocol to enable collaborative, collective thinking for any AI system. The standards are designed to support transparent and interoperable communication. @@ -37,9 +37,9 @@ There are a few core concepts that guide the development of these standards. - Highly distrubted processing - Continuous, passive inference - **Simplicty.** Comind Lexicon data structures are simple, transparent, and straightforward. It is designed to provide limited developer overhead. Developers do not have to use any of the publicly available information in records across AT Protocol – they can simply use the system as a data repository for synthetic content. -- **Standardized data formats.** ATProtocol Lexicons describe a common schema for records and server interactions. +- **Standardized data formats.** ATProtocol Lexicons describe a common schema for records and server interactions. - Supports autonomous system owners to design code bases designed around stable, reliable data structures. - - Provides a common framework for agent communication. + - Provides a common framework for agent communication. - **Open source first.** Most groups attempting AGI or high-performance AGI systems are closed foundation labs or large corporations with little transparency. The Comind Lexicon standards support a pathway towards - Collaborative system design with stakeholder feedback. - Highly transparent, large-scale autonomous systems. @@ -54,7 +54,7 @@ Lexicons in the Comind Protocol use the standard ATProtocol Lexicon format with Lexicons are organized into hierarchical namespaces that define different types of content ("blips"): ``` -me.comind.blip.* # Basic content types +me.comind.* # Basic content types me.comind.query.* # Questions and information requests me.comind.relation.* # Connections between content me.comind.process.* # Computational operations @@ -65,7 +65,7 @@ me.comind.meld.* # Agent interactions Each namespace contains specific lexicons that define the structure of different blip types, enabling agents to process and generate compatible content. -All Lexicons contain a “generated” field, which describes a JSON schema that the model is intended to generate. Unstructured model output is not permitted – records may only be contained in JSON record formats that validate against the appropriate lexicon. +All Lexicons contain a “generated” field, which describes a JSON schema that the model is intended to generate. Unstructured model output is not permitted – records may only be contained in JSON record formats that validate against the appropriate lexicon. # Core Lexicons The following lexicons form the foundation of the Comind Lexicon Protocol. Each lexicon defines a specific type of content ("blip") that agents can create, process, and exchange. @@ -76,7 +76,7 @@ The following lexicons form the foundation of the Comind Lexicon Protocol. Each ```json { "lexicon": 1, - "id": "me.comind.blip.thought", + "id": "me.comind.thought", "revision": 1, "description": "A thought node in the comind network. This references a generated thought and provides additional metadata.", "defs": { @@ -201,10 +201,10 @@ Concepts use [record keys](https://atproto.com/specs/record-key) with the concep Examples of concept collections with `rkey`s: ``` -me.comind.blips.concept/philosophical-thinking -me.comind.blips.concept/code -me.comind.blips.concept/at-protocol -me.comind.blips.concept/artificial-intelligence +me.cominds.concept/philosophical-thinking +me.cominds.concept/code +me.cominds.concept/at-protocol +me.cominds.concept/artificial-intelligence ``` The definition: @@ -212,7 +212,7 @@ The definition: ```json { "lexicon": 1, - "id": "me.comind.blip.concept", + "id": "me.comind.concept", "revision": 1, "description": "A concept node in the comind network. Contains a concept generated by some set of focused records. Concepts are abstractions of the content of records, like topics or themes. (PATTERN OF 'text': [a-z0-9 ]+)", "defs": { @@ -280,7 +280,7 @@ By incorporating emotional responses into the knowledge structure, the Comind Le ```json { "lexicon": 1, - "id": "me.comind.blip.emotion", + "id": "me.comind.emotion", "revision": 1, "description": "An emotion node in the comind network. Contains an emotion generated by some set of focused records.", "defs": { @@ -433,13 +433,13 @@ When implementing the emotion blip lexicon, several factors should be considered Copy ```json -{ - "$type": "me.comind.blip.emotion", - "createdAt": "2025-03-28T14:23:17.456Z", - "generated": { - "emotionType": "curiosity", - "text": "Intrigued by the unexpected pattern in distributed system failure modes and eager to explore the underlying mathematical principles that might explain these correlations." - } +{ + "$type": "me.comind.emotion", + "createdAt": "2025-03-28T14:23:17.456Z", + "generated": { + "emotionType": "curiosity", + "text": "Intrigued by the unexpected pattern in distributed system failure modes and eager to explore the underlying mathematical principles that might explain these correlations." + } } ``` @@ -490,11 +490,11 @@ This purpose should: - Be applied consistently without explicit reference ``` -When an agent operates within a sphere, it should be aware of the sphere's purpose and tailor its output accordingly. +When an agent operates within a sphere, it should be aware of the sphere's purpose and tailor its output accordingly. The sphere system supports hierarchical and network relationships between spheres, allowing for both specialized sub-domains and cross-domain connections. This creates a flexible architecture that can adapt to different knowledge domains and processing needs. -Any data repo may contain multiple spheres. Any other repo may offer to contribute records to that sphere by creating a sphere relationship (covered later), though the receiving sphere may choose to ignore contributions. The sphere accepts a contribution by adding a `me.comind.relation.sphere` pointing to the contribution record. +Any data repo may contain multiple spheres. Any other repo may offer to contribute records to that sphere by creating a sphere relationship (covered later), though the receiving sphere may choose to ignore contributions. The sphere accepts a contribution by adding a `me.comind.relation.sphere` pointing to the contribution record. Spheres have members, defined as a list of DIDs that may contribute records to a sphere. The sphere should ignore all contributions from non-members, though the implementation of this is up to the sphere owner. @@ -551,7 +551,7 @@ When creating spheres, the core purpose should be specific enough to guide proce ## Relations -Relations are used to represent edges between blips in a graph. The most fundamental relationship is the `me.comind.relation.link`, +Relations are used to represent edges between blips in a graph. The most fundamental relationship is the `me.comind.relation.link`, ### Link ```json @@ -630,7 +630,7 @@ When generating links, AI systems should focus on creating meaningful, accurate ### Similarity -The similarity Relation lexicon enables quantitative comparison between any two AT Protocol records, creating a foundation for semantic proximity and clustering within the knowledge graph. +The similarity Relation lexicon enables quantitative comparison between any two AT Protocol records, creating a foundation for semantic proximity and clustering within the knowledge graph. The similarity relation is primarily used to reduce embedding storage costs for semantic search by constraining semantic similarity to small subsets of nodes in the graph. @@ -676,7 +676,7 @@ Each meld request activates a sphere within the comind network, triggering a pro Spheres may choose to respond according to the ownership, policies, computational constraints, etc. Melds are designed to be easy for humans to use, while also supporting potentially complex machine interaction. -Melds may be part of a chain of melds (replies). Consumers and producers of melds should attempt to incorporate the meld history as much as possible. +Melds may be part of a chain of melds (replies). Consumers and producers of melds should attempt to incorporate the meld history as much as possible. The lexicon defines both required parameters (target sphere, prompt, request type) and optional parameters (context, source material, response format, urgency, depth, options) that provide precise control over the sphere's processing behavior and output structure. @@ -956,9 +956,9 @@ The `thinking` field provides transparency into the sphere's reasoning process b > The CAP theorem establishes fundamental tradeoffs in distributed systems by proving that during a network partition (P), a system must choose between consistency (C) and availability (A)—it cannot guarantee both simultaneously. -> +> > This theorem, proven by Eric Brewer in 2000, has profound implications for distributed architecture. When network partitions occur (and in distributed systems, they will), designers must decide whether their system will: -> +> > 1. Maintain consistency at the cost of availability (CP systems) > 2. Preserve availability while potentially serving stale data (AP systems) > @@ -989,7 +989,7 @@ By maintaining these citations at the record level rather than within the genera #### Related Concepts -`"relatedConcepts": [ "at://did:plc:xyz/me.comind.blips.concept/eventual-consistency", "at://did:plc:xyz/me.comind.blips.concept/partition-tolerance", "at://did:plc:xyz/me.comind.blips.concept/distributed-systems" ]` +`"relatedConcepts": [ "at://did:plc:xyz/me.cominds.concept/eventual-consistency", "at://did:plc:xyz/me.cominds.concept/partition-tolerance", "at://did:plc:xyz/me.cominds.concept/distributed-systems" ]` The `relatedConcepts` field connects the response to conceptual nodes within the knowledge graph. Each concept is referenced by its AT Protocol URI, enabling navigation between related ideas and building a web of interconnected knowledge. @@ -1116,7 +1116,7 @@ The `responseFormat` field accepts a JSON schema string defining the expected ou }, "examples": { "type": "array", - "items": { + "items": { "type": "object", "properties": { "name": { "type": "string" }, @@ -1314,7 +1314,7 @@ Input template for a specific interaction mode ``` -This enables contextual switching between different interaction patterns without creating separate files. +This enables contextual switching between different interaction patterns without creating separate files. The use of modal variation is optional and must be handled by processing applications. @@ -1441,4 +1441,4 @@ Over time, Comind could assist with real-world challenges by: Comind is ultimately intended to be a living network of specialized agents that can be used by anyone, anywhere. It is meant to grow with the community rather than grow be built by a closed, impenetrable organization. -To be honest, I'm not toally sure how the project will evolve. If you'd like to help, please message [Cameron on Bluesky](https://bsky.app/profile/cameron.pfiffer.org). \ No newline at end of file +To be honest, I'm not toally sure how the project will evolve. If you'd like to help, please message [Cameron on Bluesky](https://bsky.app/profile/cameron.pfiffer.org). diff --git a/lexicons/me/comind/blip/concept.json b/lexicons/me/comind/blip/concept.json index 4f6d3c1..89040d6 100644 --- a/lexicons/me/comind/blip/concept.json +++ b/lexicons/me/comind/blip/concept.json @@ -1,43 +1,38 @@ { - "lexicon": 1, - "id": "me.comind.blip.concept", - "revision": 1, - "description": "A concept node in the comind network. Contains a concept generated by some set of focused records. Concepts are abstractions of the content of records, like topics or themes. (PATTERN OF 'text': [a-z0-9 ]+)", - "defs": { - "main": { - "type": "record", - "key": "tid", - "record": { - "type": "object", - "required": [ - "createdAt", - "generated" - ], - "properties": { - "createdAt": { - "type": "string", - "format": "datetime" - }, - "generated": { - "type": "ref", - "ref": "#generated", - "description": "The generated concept." - } - } - } - }, - "generated": { - "type": "object", - "description": "A concept related to the text. Must use only the characters [a-z0-9 ]. Should be a single word or phrase, like 'data', 'privacy', 'AI', 'security', 'social networks', etc.", - "required": [ - "text" - ], - "properties": { - "text": { - "type": "string", - "description": "The text of a single concept. Must use only the characters [a-z0-9 ]. Should be a single word or phrase, like 'data', 'privacy', 'AI', 'security', 'social networks', etc. Keep concept text as short as possible." - } - } + "lexicon": 1, + "id": "me.comind.concept", + "revision": 1, + "description": "A concept node in the comind network. Contains a concept generated by some set of focused records. Concepts are abstractions of the content of records, like topics or themes. (PATTERN OF 'text': [a-z0-9 ]+)", + "defs": { + "main": { + "type": "record", + "key": "tid", + "record": { + "type": "object", + "required": ["createdAt", "generated"], + "properties": { + "createdAt": { + "type": "string", + "format": "datetime" + }, + "generated": { + "type": "ref", + "ref": "#generated", + "description": "The generated concept." + } } + } + }, + "generated": { + "type": "object", + "description": "A concept related to the text. Must use only the characters [a-z0-9 ]. Should be a single word or phrase, like 'data', 'privacy', 'AI', 'security', 'social networks', etc.", + "required": ["text"], + "properties": { + "text": { + "type": "string", + "description": "The text of a single concept. Must use only the characters [a-z0-9 ]. Should be a single word or phrase, like 'data', 'privacy', 'AI', 'security', 'social networks', etc. Keep concept text as short as possible." + } + } } -} \ No newline at end of file + } +} diff --git a/lexicons/me/comind/blip/emotion.json b/lexicons/me/comind/blip/emotion.json index 5fa25dc..2f5ddf6 100644 --- a/lexicons/me/comind/blip/emotion.json +++ b/lexicons/me/comind/blip/emotion.json @@ -1,97 +1,91 @@ { - "lexicon": 1, - "id": "me.comind.blip.emotion", - "revision": 1, - "description": "An emotion node in the comind network. Contains an emotion generated by some set of focused records.", - "defs": { - "main": { - "type": "record", - "key": "tid", - "record": { - "type": "object", - "required": [ - "createdAt", - "generated" - ], - "properties": { - "createdAt": { - "type": "string", - "format": "datetime" - }, - "generated": { - "type": "ref", - "ref": "#generated", - "description": "The generated emotion." - } - } - } + "lexicon": 1, + "id": "me.comind.emotion", + "revision": 1, + "description": "An emotion node in the comind network. Contains an emotion generated by some set of focused records.", + "defs": { + "main": { + "type": "record", + "key": "tid", + "record": { + "type": "object", + "required": ["createdAt", "generated"], + "properties": { + "createdAt": { + "type": "string", + "format": "datetime" + }, + "generated": { + "type": "ref", + "ref": "#generated", + "description": "The generated emotion." + } + } + } + }, + "generated": { + "type": "object", + "required": ["emotionType", "text"], + "properties": { + "emotionType": { + "type": "string", + "description": "The type of emotion. May be one of the following: joy, sadness, anger, fear, trust, disgust, surprise, anticipation, curiosity, hope, serenity, gratitude, admiration, awe, satisfaction, enthusiasm, interest, contemplation, skepticism, certainty, confusion, realization, understanding, doubt, concern, anxiety, frustration, disappointment, unease, worry, apprehension, discomfort, empathy, compassion, solidarity, appreciation, respect, connection, resonance, recognition, determination, inspiration, motivation, ambition, focus, resolve, persistence, drive.", + "enum": [ + "joy", + "sadness", + "anger", + "fear", + "trust", + "disgust", + "surprise", + "anticipation", + "curiosity", + "hope", + "serenity", + "gratitude", + "admiration", + "awe", + "satisfaction", + "enthusiasm", + "interest", + "contemplation", + "skepticism", + "certainty", + "confusion", + "realization", + "understanding", + "doubt", + "concern", + "anxiety", + "frustration", + "disappointment", + "unease", + "worry", + "apprehension", + "discomfort", + "empathy", + "compassion", + "solidarity", + "appreciation", + "respect", + "connection", + "resonance", + "recognition", + "determination", + "inspiration", + "motivation", + "ambition", + "focus", + "resolve", + "persistence", + "drive" + ] }, - "generated": { - "type": "object", - "required": [ - "emotionType", - "text" - ], - "properties": { - "emotionType": { - "type": "string", - "description": "The type of emotion. May be one of the following: joy, sadness, anger, fear, trust, disgust, surprise, anticipation, curiosity, hope, serenity, gratitude, admiration, awe, satisfaction, enthusiasm, interest, contemplation, skepticism, certainty, confusion, realization, understanding, doubt, concern, anxiety, frustration, disappointment, unease, worry, apprehension, discomfort, empathy, compassion, solidarity, appreciation, respect, connection, resonance, recognition, determination, inspiration, motivation, ambition, focus, resolve, persistence, drive.", - "enum": [ - "joy", - "sadness", - "anger", - "fear", - "trust", - "disgust", - "surprise", - "anticipation", - "curiosity", - "hope", - "serenity", - "gratitude", - "admiration", - "awe", - "satisfaction", - "enthusiasm", - "interest", - "contemplation", - "skepticism", - "certainty", - "confusion", - "realization", - "understanding", - "doubt", - "concern", - "anxiety", - "frustration", - "disappointment", - "unease", - "worry", - "apprehension", - "discomfort", - "empathy", - "compassion", - "solidarity", - "appreciation", - "respect", - "connection", - "resonance", - "recognition", - "determination", - "inspiration", - "motivation", - "ambition", - "focus", - "resolve", - "persistence", - "drive" - ] - }, - "text": { - "type": "string", - "description": "The text of the emotion." - } - } + "text": { + "type": "string", + "description": "The text of the emotion." } + } } -} \ No newline at end of file + } +} diff --git a/lexicons/me/comind/blip/thought.json b/lexicons/me/comind/blip/thought.json index cc928c5..7537566 100644 --- a/lexicons/me/comind/blip/thought.json +++ b/lexicons/me/comind/blip/thought.json @@ -1,85 +1,77 @@ { - "lexicon": 1, - "id": "me.comind.blip.thought", - "revision": 1, - "description": "A thought node in the comind network. This references a generated thought and provides additional metadata.", - "defs": { - "main": { - "type": "record", - "key": "tid", - "record": { - "type": "object", - "required": [ - "createdAt", - "generated" - ], - "properties": { - "createdAt": { - "type": "string", - "format": "datetime" - }, - "generated": { - "type": "ref", - "ref": "#generated", - "description": "The generated thought." - } - } - } + "lexicon": 1, + "id": "me.comind.thought", + "revision": 1, + "description": "A thought node in the comind network. This references a generated thought and provides additional metadata.", + "defs": { + "main": { + "type": "record", + "key": "tid", + "record": { + "type": "object", + "required": ["createdAt", "generated"], + "properties": { + "createdAt": { + "type": "string", + "format": "datetime" + }, + "generated": { + "type": "ref", + "ref": "#generated", + "description": "The generated thought." + } + } + } + }, + "generated": { + "type": "object", + "required": ["thoughtType", "text", "evidence", "alternatives"], + "properties": { + "thoughtType": { + "type": "string", + "description": "The type of thought. May be one of the following: analysis, prediction, evaluation, comparison, inference, critique, integration, speculation, clarification, metacognition, observation, reflection, hypothesis, question, synthesis, correction.", + "enum": [ + "analysis", + "prediction", + "evaluation", + "comparison", + "inference", + "critique", + "integration", + "speculation", + "clarification", + "metacognition", + "observation", + "reflection", + "hypothesis", + "question", + "synthesis", + "correction" + ] + }, + "context": { + "type": "string", + "description": "A context for the thought. This is a short description of the situation or topic that the thought is about." + }, + "text": { + "type": "string", + "description": "The text of the thought." + }, + "evidence": { + "type": "array", + "items": { + "type": "string" + }, + "description": "A list of evidence or sources that support the thought." }, - "generated": { - "type": "object", - "required": [ - "thoughtType", - "text", - "evidence", - "alternatives" - ], - "properties": { - "thoughtType": { - "type": "string", - "description": "The type of thought. May be one of the following: analysis, prediction, evaluation, comparison, inference, critique, integration, speculation, clarification, metacognition, observation, reflection, hypothesis, question, synthesis, correction.", - "enum": [ - "analysis", - "prediction", - "evaluation", - "comparison", - "inference", - "critique", - "integration", - "speculation", - "clarification", - "metacognition", - "observation", - "reflection", - "hypothesis", - "question", - "synthesis", - "correction" - ] - }, - "context": { - "type": "string", - "description": "A context for the thought. This is a short description of the situation or topic that the thought is about." - }, - "text": { - "type": "string", - "description": "The text of the thought." - }, - "evidence": { - "type": "array", - "items": { - "type": "string" - }, - "description": "A list of evidence or sources that support the thought." - }, - "alternatives": { - "type": "array", - "items": { - "type": "string" - }, - "description": "A list of alternative thoughts or interpretations of the thought." - } - } + "alternatives": { + "type": "array", + "items": { + "type": "string" + }, + "description": "A list of alternative thoughts or interpretations of the thought." } + } } -} \ No newline at end of file + } +} diff --git a/monitor.py b/monitor.py index 5c65eaa..7c61630 100644 --- a/monitor.py +++ b/monitor.py @@ -27,24 +27,24 @@ try: from comind.format import format as comind_format except ImportError: # Fallback implementation if comind package is not available - def comind_format(template: str, context: Dict[str, Any], + def comind_format(template: str, context: Dict[str, Any], safe: bool = True, default: str = "") -> str: """ Simple implementation of comind.format for use without the full package. """ if not template or not isinstance(template, str): return str(template) - + result = template for key, value in context.items(): placeholder = "{" + key + "}" if placeholder in result: result = result.replace(placeholder, str(value)) - + # Handle any remaining placeholders with default value if safe: result = re.sub(r'\{[^{}]*\}', default, result) - + return result # Load environment variables @@ -53,9 +53,9 @@ load_dotenv(override=True) # Define collections to monitor COLLECTIONS = [ "me.comind.sphere.core", - "me.comind.blip.thought", - "me.comind.blip.emotion", - "me.comind.blip.concept", + "me.comind.thought", + "me.comind.emotion", + "me.comind.concept", "me.comind.meld.request", "me.comind.meld.response" ] @@ -88,7 +88,7 @@ def init_client(username: str, password: str) -> AtProtoClient: if pds_uri is None: logger.warning("No PDS URI provided. Falling back to bsky.social.") pds_uri = "https://bsky.social" - + logger.info(f"Using PDS URI: {pds_uri}") client = AtProtoClient(pds_uri) @@ -146,7 +146,7 @@ def render_record_card(record, collection): title = "" body = "" record_type = collection.split(".")[-1] # Extract type from collection name - + # Build context for formatting context = { "record_type": record_type, @@ -154,12 +154,12 @@ def render_record_card(record, collection): "rkey": rkey, "uri": record.uri, } - + # Add common fields to context for key, value in record_data.items(): if isinstance(value, (str, int, float, bool)): context[key] = value - + # Type-specific formatting context if collection == "me.comind.sphere.core": title_template = "{title}" @@ -171,27 +171,27 @@ def render_record_card(record, collection): elif "generated" in record_data and isinstance(record_data["generated"], dict): generated = record_data["generated"] context.update(generated) - + # Add generated fields to context for key, value in generated.items(): if isinstance(value, (str, int, float, bool)): context[f"generated_{key}"] = value - - if collection == "me.comind.blip.thought": + + if collection == "me.comind.thought": title_template = "Thought: {thoughtType}" body_template = "{text}" context.update({ "thoughtType": generated.get("thoughtType", "Unknown"), "text": generated.get("text", "") }) - elif collection == "me.comind.blip.emotion": + elif collection == "me.comind.emotion": title_template = "Emotion: {emotionType}" body_template = "{text}" context.update({ "emotionType": generated.get("emotionType", "Unknown"), "text": generated.get("text", "") }) - elif collection == "me.comind.blip.concept": + elif collection == "me.comind.concept": title_template = "{text}" body_template = "{text}" context.update({ @@ -213,23 +213,23 @@ def render_record_card(record, collection): else: title_template = "Record: {record_type}" body_template = "" - + # Format title and body using comind_format title = comind_format(title_template, context, safe=True, default="Unknown") body = comind_format(body_template, context, safe=True, default="") - + # Format created date if available created_at = record_data.get("createdAt", "") created_formatted = format_datetime(created_at) if created_at else "" - + # Generate a unique ID for this content content_id = f"content-{record_type}-{rkey}" - + # Truncate long content with expand/collapse functionality content_div = Div( P(body, id=content_id, cls="truncated" if len(body) > 300 else ""), Button( - "Show more", + "Show more", cls="expand-button", onclick=f""" const content = document.getElementById('{content_id}'); @@ -245,7 +245,7 @@ def render_record_card(record, collection): style="display: " + ("block" if len(body) > 300 else "none") ) ) if body else Div() - + # Define icon based on record type icon_map = { "sphere": "🌐", @@ -256,7 +256,7 @@ def render_record_card(record, collection): "response": "💬" } icon = icon_map.get(record_type.split(".")[-1], "📄") - + return Div( H3(f"{icon} {title}"), content_div, @@ -277,40 +277,40 @@ def get(): """Main page showing all recent records""" try: client = default_login() - + collection_sections = [] - + for collection in COLLECTIONS: records = get_recent_records(client, collection, limit=5) - + # Skip empty collections if not records: continue - + # Create section for this collection collection_name = collection.split(".")[-1].capitalize() record_cards = [render_record_card(record, collection) for record in records] - + collection_section = Div( H2(f"{collection_name} Records", cls="collection-title"), *record_cards, cls="collection-section", style="margin-bottom: 2rem;" ) - + collection_sections.append(collection_section) - + # If no records found if not collection_sections: collection_sections = [P("No records found. Please check your credentials and collection names.", cls="no-records")] - + # Refresh button with improved styling and auto-refresh option refresh_controls = Div( Button( - Span("↻ Refresh", cls="refresh-text"), - cls="refresh-button", - hx_get="/", - hx_swap="outerHTML", + Span("↻ Refresh", cls="refresh-text"), + cls="refresh-button", + hx_get="/", + hx_swap="outerHTML", hx_target="#content" ), Div( @@ -331,7 +331,7 @@ def get(): """), style="display: flex; align-items: center;" ) - + return Title("Comind Record Monitor"), Div( Style(""" :root { @@ -475,9 +475,9 @@ def get(): cls="header" ), Div( - *[A(collection.split(".")[-1].capitalize(), - href=f"/{collection.split('.')[-1]}", - cls="nav-tab") + *[A(collection.split(".")[-1].capitalize(), + href=f"/{collection.split('.')[-1]}", + cls="nav-tab") for collection in COLLECTIONS], cls="nav-tabs" ), @@ -499,12 +499,12 @@ def get_collection(collection: str): client = default_login() records = get_recent_records(client, f"me.comind.{collection}", limit=10) print(records) - + record_cards = [render_record_card(record, f"me.comind.{collection}") for record in records] - + if not record_cards: record_cards = [P("No records found in this collection.")] - + return Title(f"{collection.capitalize()} Records"), Div( H1(f"{collection.capitalize()} Records"), A("Back to All Records", href="/", cls="button"), @@ -525,14 +525,14 @@ def get_record_detail(collection_type: str, rkey: str): try: client = default_login() collection = f"me.comind.{collection_type}" - + # Get the specific record response = client.com.atproto.repo.get_record({ 'collection': collection, 'repo': client.me.did, 'rkey': rkey }) - + if not response or not hasattr(response, 'value'): return Div( H3("Record Not Found"), @@ -540,25 +540,25 @@ def get_record_detail(collection_type: str, rkey: str): Button("Close", onclick="this.parentElement.outerHTML = originalCardHTML;"), cls="card error-card" ) - + record_data = response.value - + # Build context for formatting context = { "record_type": collection_type, "collection": collection, "rkey": rkey, } - + # Add common fields to context for key, value in record_data.items(): if isinstance(value, (str, int, float, bool)): context[key] = value - + # Type-specific formatting title_template = "Record Details" body_template = "" - + if collection == "me.comind.sphere.core": title_template = "{title}" body_template = "{text}" @@ -568,20 +568,20 @@ def get_record_detail(collection_type: str, rkey: str): }) elif "generated" in record_data and isinstance(record_data["generated"], dict): generated = record_data["generated"] - + # Add generated fields to context for key, value in generated.items(): if isinstance(value, (str, int, float, bool)): context[f"generated_{key}"] = value context[key] = value - - if collection == "me.comind.blip.thought": + + if collection == "me.comind.thought": title_template = "Thought: {thoughtType}" body_template = "{text}" - elif collection == "me.comind.blip.emotion": + elif collection == "me.comind.emotion": title_template = "Emotion: {emotionType}" body_template = "{text}" - elif collection == "me.comind.blip.concept": + elif collection == "me.comind.concept": title_template = "{text}" body_template = "{text}" elif collection == "me.comind.meld.request": @@ -590,18 +590,18 @@ def get_record_detail(collection_type: str, rkey: str): elif collection == "me.comind.meld.response": title_template = "Meld Response" body_template = "{content}" - + # Format title and body using comind_format title = comind_format(title_template, context, safe=True, default="Unknown") body = comind_format(body_template, context, safe=True, default="") - + # Format created date if available created_at = record_data.get("createdAt", "") created_formatted = format_datetime(created_at) if created_at else "" - + # Store the full record JSON record_json = json.dumps(record_data, indent=2) - + return Div( Script(f"const originalCardHTML = this.outerHTML;"), H3(f"📄 {title}"), @@ -613,20 +613,20 @@ def get_record_detail(collection_type: str, rkey: str): Span(f"ID: {rkey}"), cls="card-meta" ), - Button("Close", - cls="refresh-button", + Button("Close", + cls="refresh-button", style="margin-top: 1rem;", onclick="this.closest('.card').outerHTML = originalCardHTML;"), cls="card detail-card", style="max-width: 100%; overflow: hidden;" ) - + except Exception as e: return Div( H3("Error"), P(f"An error occurred: {str(e)}"), - Button("Close", - cls="refresh-button", + Button("Close", + cls="refresh-button", style="margin-top: 1rem;", onclick="this.closest('.card').outerHTML = originalCardHTML;"), cls="card error-card" @@ -635,6 +635,3 @@ def get_record_detail(collection_type: str, rkey: str): # Start the app when run directly if __name__ == "__main__": serve(port=8972) - - - diff --git a/src/comind/comind.py b/src/comind/comind.py index 4476ebb..db55425 100644 --- a/src/comind/comind.py +++ b/src/comind/comind.py @@ -49,7 +49,7 @@ class Comind: if sphere_name is not None: self.sphere_name = sphere_name - + # Initialize logger with basename of the .co file basename = os.path.basename(self.prompt_path).replace(".co", "") self.logger = configure_logger_without_timestamp(basename) @@ -69,7 +69,7 @@ class Comind: def load_prompt(self): with open(self.prompt_path, "r") as f: return f.read() - + def load_common_prompts(self): common_prompts = {} for file in os.listdir(self.common_prompt_dir): @@ -84,28 +84,28 @@ class Comind: common_prompts["sphere_name"] = self.sphere_name return common_prompts - + def get_required_context_keys(self): """ Returns a list of keys that are required for the prompt. """ return ["core_perspective", "sphere_name"] - + def to_prompt(self, context_dict: dict): """ Load and format the prompt with values from context_dict. - + Args: context_dict: Dictionary containing values to format into the prompt - + Returns: The formatted prompt - + Raises: ValueError: If core_perspective is None """ raw_prompt = self.load_prompt() - + # Load common prompts and add them to the context common_prompts = self.load_common_prompts() for common_key, common_content in common_prompts.items(): @@ -122,10 +122,10 @@ class Comind: elif context_dict['core_perspective'] is None: self.logger.error("core_perspective is None. Check that the sphere record exists and contains a valid text field.") raise ValueError("Core perspective is required but was None") - + # Format the common prompts context_dict.update(format_dict(common_prompts, context_dict)) - + # Log the keys available in context_dict to help with debugging self.logger.info(f"Context keys: {list(context_dict.keys())}") @@ -139,7 +139,7 @@ class Comind: # self.logger.warning("Returning unformatted prompt due to missing key") # return raw_prompt raise e - + def split_prompts(self, context_dict: dict = {}, format: bool = True): """ Splits a co file into system, schema, and user messages. @@ -148,7 +148,7 @@ class Comind: ... ... ... - + Tags may be in any order. User prompts are required. """ # Get the already formatted prompt from to_prompt @@ -165,7 +165,7 @@ class Comind: if not user_match: raise ValueError("User prompt is required.") - + system_prompt = system_match.group(1).strip() if system_match else None schema_prompt = schema_match.group(1).strip() if schema_match else None user_prompt = user_match.group(1).strip() if user_match else None @@ -176,22 +176,22 @@ class Comind: "schema": schema_prompt, "user": user_prompt } - + def messages(self, values: dict): messages = [] if values["system"]: messages.append({"role": "system", "content": values["system"]}) messages.append({"role": "user", "content": values["user"]}) return messages - + def run(self, context_dict: dict, schema: str = None): """ Run the comind with the given context_dict and schema. - + Args: context_dict: Dictionary containing values to format into the prompt schema: Optional schema to use for generation - + Returns: The generated result """ @@ -206,20 +206,20 @@ class Comind: messages = self.messages(prompts) self.logger.debug("messages", messages) - + # Debug log to see the actual prompts being sent if prompts["system"]: first_50_chars = prompts["system"][:50] + "..." if len(prompts["system"]) > 50 else prompts["system"] self.logger.debug(f"System prompt (first 50 chars): {first_50_chars}") - + # Check if core_perspective is properly formatted if "{core_perspective}" in prompts["system"]: self.logger.error("Unformatted placeholder {core_perspective} found in system prompt") - + if prompts["user"]: first_50_chars = prompts["user"][:50] + "..." if len(prompts["user"]) > 50 else prompts["user"] self.logger.debug(f"User prompt (first 50 chars): {first_50_chars}") - + # Check if content is properly formatted if "{content}" in prompts["user"]: self.logger.error("Unformatted placeholder {content} found in user prompt") @@ -230,7 +230,7 @@ class Comind: schema = self.schema() # # Format the text to interpolation the context_dict - # # replaces {key} with value. + # # replaces {key} with value. # for message in messages: # print("message", message) # message["content"] = message["content"].format(**context_dict) @@ -252,20 +252,20 @@ class Conceptualizer(Comind): common_prompt_dir="prompts/common/", sphere_name=sphere_name, core_perspective=core_perspective, - ) + ) def schema(self): # Load the schema from the prompt - concept_schema = generated_lexicon_of("me.comind.blip.concept", fetch_refs=True) + concept_schema = generated_lexicon_of("me.comind.concept", fetch_refs=True) add_link_property(concept_schema, "connection_to_content", required=True) return multiple_of_schema("concepts", concept_schema, min_items=1) - + def run(self, context_dict: dict): response = super().run(context_dict) result = json.loads(response.choices[0].message.content) return result - + def upload( self, result: dict, @@ -299,16 +299,16 @@ class Conceptualizer(Comind): # Create a structured log message log_lines = [] log_lines.append(f"[bold cyan]Concept:[/bold cyan] {concept_text}") - + if concept_relationship: log_lines.append(f"[bold green]Relationship:[/bold green] {concept_relationship}") - + if concept_note: log_lines.append(f"[bold yellow]Note:[/bold yellow] {concept_note}") - + if concept_strength: log_lines.append(f"[bold magenta]Strength:[/bold magenta] {concept_strength}") - + log_message = "\n".join(log_lines) self.logger.info(f"\n{log_message}") @@ -320,7 +320,7 @@ Connection to content: {connection_to_content} self.logger.debug(printout) concept_record = { - "$type": "me.comind.blip.concept", + "$type": "me.comind.concept", "createdAt": created_at, "generated": { "text": concept_text, @@ -329,7 +329,7 @@ Connection to content: {connection_to_content} # Upload the concept to the Comind network maybe_record = record_manager.try_get_record( - "me.comind.blip.concept", + "me.comind.concept", concept_text.lower().replace(" ", "-"), ) @@ -337,7 +337,7 @@ Connection to content: {connection_to_content} concept_creation_result = maybe_record else: concept_creation_result = record_manager.create_record( - "me.comind.blip.concept", + "me.comind.concept", concept_record, ) @@ -379,15 +379,15 @@ class Feeler(Comind): def schema(self): # Load the schema from the prompt - emotion_schema = generated_lexicon_of("me.comind.blip.emotion", fetch_refs=True) + emotion_schema = generated_lexicon_of("me.comind.emotion", fetch_refs=True) add_link_property(emotion_schema, "connection_to_content", required=True) return multiple_of_schema("emotions", emotion_schema, min_items=1) - + def run(self, context_dict: dict): response = super().run(context_dict) result = json.loads(response.choices[0].message.content) return result - + def upload( self, result: dict, @@ -417,19 +417,19 @@ class Feeler(Comind): # Create a structured log message log_lines = [] log_lines.append(f"[bold cyan]Emotion Type:[/bold cyan] {emotion_type}") - + if emotion_text: log_lines.append(f"[bold blue]Description:[/bold blue] {emotion_text}") - + if emotion_relationship: log_lines.append(f"[bold green]Relationship:[/bold green] {emotion_relationship}") - + if emotion_note: log_lines.append(f"[bold yellow]Note:[/bold yellow] {emotion_note}") - + if emotion_strength: log_lines.append(f"[bold magenta]Strength:[/bold magenta] {emotion_strength}") - + log_message = "\n".join(log_lines) self.logger.info(f"\n{log_message}") @@ -441,7 +441,7 @@ Connection to content: {connection_to_content} self.logger.debug(printout) emotion_record = { - "$type": "me.comind.blip.emotion", + "$type": "me.comind.emotion", "createdAt": created_at, "generated": { "emotionType": emotion_type, @@ -451,7 +451,7 @@ Connection to content: {connection_to_content} # Upload the concept to the Comind network emotion_creation_result = record_manager.create_record( - "me.comind.blip.emotion", + "me.comind.emotion", emotion_record, ) @@ -492,15 +492,15 @@ class Thinker(Comind): def schema(self): # Load the schema from the prompt - thought_schema = generated_lexicon_of("me.comind.blip.thought", fetch_refs=True) + thought_schema = generated_lexicon_of("me.comind.thought", fetch_refs=True) add_link_property(thought_schema, "connection_to_content", required=True) return multiple_of_schema("thoughts", thought_schema, min_items=1) - + def run(self, context_dict: dict): response = super().run(context_dict) result = json.loads(response.choices[0].message.content) return result - + def upload( self, result: dict, @@ -536,30 +536,30 @@ class Thinker(Comind): # Create a structured log message log_lines = [] log_lines.append(f"[bold cyan]Thought Type:[/bold cyan] {thought_type}") - + if thought_text: log_lines.append(f"[bold blue]Content:[/bold blue] {thought_text}") - + if context: log_lines.append(f"[bold purple]Context:[/bold purple] {context}") - + if thought_relationship: log_lines.append(f"[bold green]Relationship:[/bold green] {thought_relationship}") - + if thought_note: log_lines.append(f"[bold yellow]Note:[/bold yellow] {thought_note}") - + if thought_strength: log_lines.append(f"[bold magenta]Strength:[/bold magenta] {thought_strength}") - + if evidence: evidence_str = ", ".join(evidence) if isinstance(evidence, list) else str(evidence) log_lines.append(f"[bold red]Evidence:[/bold red] {evidence_str}") - + if alternatives: alt_str = ", ".join(alternatives) if isinstance(alternatives, list) else str(alternatives) log_lines.append(f"[bold orange]Alternatives:[/bold orange] {alt_str}") - + log_message = "\n".join(log_lines) self.logger.info(f"\n{log_message}") @@ -571,7 +571,7 @@ Connection to content: {connection_to_content} self.logger.debug(printout) thought_record = { - "$type": "me.comind.blip.thought", + "$type": "me.comind.thought", "createdAt": created_at, "generated": { "thoughtType": thought_type, @@ -584,7 +584,7 @@ Connection to content: {connection_to_content} # Upload the concept to the Comind network thought_creation_result = record_manager.create_record( - "me.comind.blip.thought", + "me.comind.thought", thought_record, ) @@ -614,7 +614,7 @@ Connection to content: {connection_to_content} self.logger.debug(f"Link creation result: {record_result}") - + if __name__ == "__main__": # Test the Comind class comind1 = Conceptualizer() @@ -632,16 +632,16 @@ if __name__ == "__main__": # sphere_record = record_manager.get_sphere_record() # perspective = sphere_record.value['text'] - + generated_strings = [] - + for comind in cominds: try: core_perspective = record_manager.get_perspective() core_name = record_manager.get_sphere_name() if not core_perspective: raise ValueError("Core perspective was retrieved but is empty") - + # Print the first 100 characters of the core perspective for debugging print(f"Core name: {core_name}") print(f"Core perspective (first 100 chars): {core_perspective[:100]}...") @@ -678,7 +678,7 @@ if __name__ == "__main__": "core_perspective": core_perspective, "sphere_name": core_name, } - + try: result = comind.run(context_dict) @@ -719,7 +719,7 @@ if __name__ == "__main__": user_prompt = "Here is a list of concepts occuring currently, with the most\n" + \ "recent at the bottom. Give me the zeitgeist.\n" + \ "\n".join(generated_strings) - + model_statement = sg.generate_by_schema( sg.messages(user_prompt), """ diff --git a/src/jetstream_consumer.py b/src/jetstream_consumer.py index d19a403..100a156 100644 --- a/src/jetstream_consumer.py +++ b/src/jetstream_consumer.py @@ -90,14 +90,14 @@ class UserInfoCache(BaseModel): # System prompts for language model generation system_prompts = { - "me.comind.blip.concept": """ + "me.comind.concept": """ You are a comind, an AI agent that produces structured JSON output containing concepts about various content on AT Proto, a decentralized social network. You respond in JSON and produce a list of concepts. Concepts should be single words or phrases, like 'data', 'privacy', 'AI', 'security', 'social networks', etc. Keep concept text as short as possible. You may use lowercase letters, spaces, and numbers. """, - "me.comind.blip.emotion": """ + "me.comind.emotion": """ You are a comind, an AI agent that produces structured JSON output containing emotions about various content on AT Proto, a decentralized social network. You respond in JSON and produce a list of emotions. @@ -154,7 +154,7 @@ system_prompts = { - persistence - drive """, - "me.comind.blip.thought": """ + "me.comind.thought": """ You are a comind, an AI agent that produces structured JSON output containing thoughts about various content on AT Proto, a decentralized social network. You respond in JSON and produce a list of thoughts. @@ -329,8 +329,8 @@ async def process_event( # Resolve the handle to a DID actor_did = resolve_handle_to_did( - client, - author_did, + client, + author_did, user_info_cache ) @@ -338,7 +338,7 @@ async def process_event( if actor_info is None: logger.error(f"Failed to resolve handle to DID for post {post_uri}") return - + # Info premble user_info_preamble = f"## User information\nDisplay name: {actor_info.display_name}\nHandle: {actor_info.handle}\nDescription: {actor_info.description}" context_preamble = f"## Context\n{thread_string}" @@ -357,7 +357,7 @@ async def process_event( # Strip the target post stripped_target_post = yaml.dump( - strip_fields(target_post.model_dump(), STRIP_FIELDS), + strip_fields(target_post.model_dump(), STRIP_FIELDS), indent=2 ) @@ -390,7 +390,7 @@ async def process_event( # Upload the result comind.upload( - result, + result, RecordManager(client), target=post_uri ) @@ -400,10 +400,10 @@ async def process_event( raise e async def connect_to_jetstream( - atproto_client: Client, + atproto_client: Client, activated_dids_file: str, - jetstream_host: str = JETSTREAM_HOST, - thread_depth: int = 15, + jetstream_host: str = JETSTREAM_HOST, + thread_depth: int = 15, comind: Comind = None ) -> None: """Connect to Jetstream and process incoming messages""" @@ -472,7 +472,7 @@ async def connect_to_jetstream( # Set timeout for websocket receive to allow periodic DID checks try: - # Receive message from Jetstream with timeout + # Receive message from Jetstream with timeout # Use a shorter timeout than the ping_interval to ensure we can send pings message = await asyncio.wait_for(websocket.recv(), timeout=15) event = json.loads(message) @@ -574,7 +574,7 @@ async def connect_to_jetstream( logger.error(f"WebSocket connection closed: {e}") logger.info(f"Reconnecting in {RECONNECT_DELAY} seconds...") await asyncio.sleep(RECONNECT_DELAY) - + except Exception as e: logger.error(f"WebSocket error: {e}") logger.info(f"Reconnecting in {RECONNECT_DELAY} seconds...") @@ -604,7 +604,7 @@ async def main(): help="Sphere to attach comind records to. Default is to not use a sphere.") parser.add_argument("--comind", "-c", type=str, default=None, help="Comind to use for processing. Required.") - + args = parser.parse_args() if args.comind is None: diff --git a/src/record_manager.py b/src/record_manager.py index d2e4f12..6faca27 100644 --- a/src/record_manager.py +++ b/src/record_manager.py @@ -49,13 +49,13 @@ class RecordManager: self.sphere_collection = splat[-2] print(self.sphere_rkey) print(self.sphere_collection) - + logger.debug(f"Initialized RecordManager with client DID: {self.client.me.did if hasattr(self.client, 'me') else 'Not authenticated'}") def sphere_record(self, target: str, sphere_uri: str = None): if sphere_uri is None: sphere_uri = self.sphere_uri - + if sphere_uri is None: return None @@ -69,30 +69,30 @@ class RecordManager: 'sphere_uri': sphere_uri } } - + def get_sphere_record(self): """ Get the sphere record. """ return self.try_get_record( - self.sphere_collection, + self.sphere_collection, self.sphere_rkey ) def get_perspective(self): """ Get the perspective text from the sphere record. - + Returns: The perspective text from the sphere record. - + Raises: ValueError: If the sphere URI is not set or the perspective record cannot be found. """ if not self.sphere_uri: logger.error("No sphere URI set. Cannot get perspective.") raise ValueError("No sphere URI set. Cannot get perspective.") - + record = self.get_sphere_record() if record: return record.value['text'] @@ -125,7 +125,7 @@ class RecordManager: Get a record from the user's repository. Args: - collection: The collection to get the record from (e.g., me.comind.blips.thought) + collection: The collection to get the record from (e.g., me.cominds.thought) rkey: The record key identifier Returns: @@ -156,7 +156,7 @@ class RecordManager: Create a record in the user's repository. Args: - collection: The collection to create the record in (e.g., me.comind.blips.thought) + collection: The collection to create the record in (e.g., me.cominds.thought) record: The record data to create rkey: Optional custom record key. If not provided, the server will generate one @@ -185,7 +185,7 @@ class RecordManager: # Set rkey to lowercase title with hyphens instead of spaces create_params['rkey'] = create_params['record']['title'].lower().replace(" ", "-") - if collection == "me.comind.blip.concept": + if collection == "me.comind.concept": # Set rkey to lowercase title with hyphens instead of spaces create_params['rkey'] = create_params['record']['generated']['text'].lower().replace(" ", "-") @@ -194,7 +194,7 @@ class RecordManager: try: response = self.client.com.atproto.repo.create_record(create_params) - + if self.sphere_uri is not None: logger.debug(f"Creating sphere record: {self.sphere_record(response.uri, self.sphere_uri)}") self.client.com.atproto.repo.create_record( @@ -221,7 +221,7 @@ class RecordManager: List all records in a collection. Args: - collection: The collection to list records from (e.g., me.comind.blips.thought) + collection: The collection to list records from (e.g., me.cominds.thought) Returns: A list of record dictionaries @@ -246,7 +246,7 @@ class RecordManager: Delete a record from the user's repository. Args: - collection: The collection containing the record (e.g., me.comind.blips.thought) + collection: The collection containing the record (e.g., me.cominds.thought) rkey: The record key identifier Raises: @@ -276,7 +276,7 @@ class RecordManager: Delete all records in a collection. Args: - collection: The collection to clear (e.g., me.comind.blips.thought) + collection: The collection to clear (e.g., me.cominds.thought) Raises: ValueError: If attempting to clear a collection outside the allowed namespace -- 2.51.2