Merge branch 'main' into fix/conn-id-in-keylist-webhook

Author: dbluhm

.github/workflows/pip-audit.yml

@@ -16,8 +16,9 @@ jobs:
         run: |
           python -m venv env/
           source env/bin/activate
+          python -m pip install --upgrade pip
           python -m pip install .
-      - uses: trailofbits/gh-action-pip-audit@v0.0.4
+      - uses: pypa/gh-action-pip-audit@v1.0.0
         with:
           virtual-environment: env/
           local: true

.github/workflows/sonarcloud.yml

@@ -29,7 +29,7 @@ In practice, DIDs and DID Documents are used for a variety of purposes but espec
 
 ## `DIDResolver`
 
-In ACA-Py, the `DIDResolver` provides the interface to resolve DIDs using registered method resolvers. Method resolver registration happens on startup through the `DIDResolverRegistry`. This registry enables additional resolvers to be loaded via plugin.
+In ACA-Py, the `DIDResolver` provides the interface to resolve DIDs using registered method resolvers. Method resolver registration happens on startup in a `did_resolvers` list. This registry enables additional resolvers to be loaded via plugin.
 
 #### Example usage:
 ```python=
@@ -73,17 +73,17 @@ The following is an example method resolver implementation. In this example, we
 
 ```python=
 from aries_cloudagent.config.injection_context import InjectionContext
-from aries_cloudagent.resolver.did_resolver_registry import DIDResolverRegistry
+from ..resolver.did_resolver import DIDResolver
 
 from .example_resolver import ExampleResolver
 
 
 async def setup(context: InjectionContext):
     """Setup the plugin."""
-    registry = context.inject(DIDResolverRegistry)
+    registry = context.inject(DIDResolver)
     resolver = ExampleResolver()
     await resolver.setup(context)
-    registry.register(resolver)
+    registry.append(resolver)
 ```
 
 #### `example_resolver.py`

DIDResolution.md

@@ -1,7 +1,5 @@
 # Transaction Endorser Support
 
-Note that the ACA-Py transaction support is in the process of code refactor and cleanup.  The following documents the current state, but is subject to change.
-
 ACA-Py supports an [Endorser Protocol](https://github.com/hyperledger/aries-rfcs/pull/586), that allows an un-privileged agent (an "Author") to request another agent (the "Endorser") to sign their transactions so they can write these transactions to the ledger.  This is required on Indy ledgers, where new agents will typically be granted only "Author" privileges.
 
 Transaction Endorsement is built into the protocols for Schema, Credential Definition and Revocation, and endorsements can be explicitely requested, or ACA-Py can be configured to automate the endorsement workflow.
@@ -61,3 +59,44 @@ Endorsement:
                         For Authors, specify whether to automatically promote a DID to the wallet public DID after writing to the ledger.
 ```
 
+## How Aca-py Handles Endorsements
+
+Internally, the Endorsement functionality is implemented as a protocol, and is implemented consistently with other protocols:
+
+- a [routes.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/routes.py) file exposes the admin endpoints
+- [handler files](https://github.com/hyperledger/aries-cloudagent-python/tree/main/aries_cloudagent/protocols/endorse_transaction/v1_0/handlers) implement responses to any received Endorse protocol messages
+- a [manager.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py) file implements common functionality that is called from both the routes.py and handler classes (as well as from other classes that need to interact with Endorser functionality)
+
+The Endorser makes use of the [Event Bus](https://github.com/hyperledger/aries-cloudagent-python/blob/main/CHANGELOG.md#july-14-2021) (links to the PR which links to a hackmd doc) to notify other protocols of any Endorser events of interest.  For example, after a Credential Definition endorsement is received, the TransactionManager writes the endorsed transaction to the ledger and uses the Event Bus to notify the Credential Defintition manager that it can do any required post-processing (such as writing the cred def record to the wallet, initiating the revocation registry, etc.).
+
+The overall architecture can be illustrated as:
+
+![Class Diagram](./docs/assets/endorser-design.png)
+
+### Create Credential Definition and Revocation Registry
+
+An example of an Endorser flow is as follows, showing how a credential definition endorsement is received and processed, and optionally kicks off the revocation registry process:
+
+![Sequence Diagram](./docs/assets/endorse-cred-def.png)
+
+You can see that there is a standard endorser flow happening each time there is a ledger write (illustrated in the "Endorser" process).
+
+At the end of each endorse sequence, the TransactionManager sends a notification via the EventBus so that any dependant processing can continue.  Each Router is responsible for listening and responding to these notifications if necessary.
+
+For example:
+
+- Once the credential definition is created, a revocation registry must be created (for revocable cred defs)
+- Once the revocation registry is created, a revocation entry must be created
+- Potentially, the cred def status could be updated once the revocation entry is completed
+
+Using the EventBus decouples the event sequence.  Any functions triggered by an event notification are typically also available directly via Admin endpoints.
+
+### Create DID and Promote to Public
+
+... and an example of creating a DID and promoting it to public (and creating an ATTRIB for the endpoint:
+
+![Sequence Diagram](./docs/assets/endorse-public-did.png)
+
+You can see the same endorsement processes in this sequence.
+
+Once the DID is written, the DID can (optionally) be promoted to the public DID, which will also invoke an ATTRIB transaction to write the endpoint.

Endorser.md

@@ -12,7 +12,6 @@
 from ..core.protocol_registry import ProtocolRegistry
 from ..core.goal_code_registry import GoalCodeRegistry
 from ..resolver.did_resolver import DIDResolver
-from ..resolver.did_resolver_registry import DIDResolverRegistry
 from ..tails.base import BaseTailsServer
 
 from ..protocols.actionmenu.v1_0.base_service import BaseMenuService
@@ -50,12 +49,8 @@ async def build_context(self) -> InjectionContext:
         # Global event bus
         context.injector.bind_instance(EventBus, EventBus())
 
-        # Global did resolver registry
-        did_resolver_registry = DIDResolverRegistry()
-        context.injector.bind_instance(DIDResolverRegistry, did_resolver_registry)
-
         # Global did resolver
-        context.injector.bind_instance(DIDResolver, DIDResolver(did_resolver_registry))
+        context.injector.bind_instance(DIDResolver, DIDResolver([]))
 
         await self.bind_providers(context)
         await self.load_plugins(context)

aries_cloudagent/config/default_context.py

@@ -24,7 +24,7 @@
 from ...protocols.coordinate_mediation.v1_0.models.mediation_record import (
     MediationRecord,
 )
-from ...resolver.did_resolver import DIDResolver, DIDResolverRegistry
+from ...resolver.did_resolver import DIDResolver
 from ...multitenant.base import BaseMultitenantManager
 from ...multitenant.manager import MultitenantManager
 from ...storage.base import BaseStorage
@@ -92,7 +92,7 @@ async def build_context(self) -> InjectionContext:
         context.injector.bind_instance(ProfileManager, InMemoryProfileManager())
         context.injector.bind_instance(ProtocolRegistry, ProtocolRegistry())
         context.injector.bind_instance(BaseWireFormat, self.wire_format)
-        context.injector.bind_instance(DIDResolver, DIDResolver(DIDResolverRegistry()))
+        context.injector.bind_instance(DIDResolver, DIDResolver([]))
         context.injector.bind_instance(EventBus, MockEventBus())
         return context
 

aries_cloudagent/core/tests/test_conductor.py

@@ -274,6 +274,8 @@ async def credentials_remove(request: web.BaseRequest):
         async with context.profile.session() as session:
             holder = session.inject(IndyHolder)
             await holder.delete_credential(credential_id)
+        topic = "acapy::record::credential::delete"
+        await context.profile.notify(topic, {"id": credential_id, "state": "deleted"})
     except WalletNotFoundError as err:
         raise web.HTTPNotFound(reason=err.roll_up) from err
 
@@ -376,6 +378,10 @@ async def w3c_cred_remove(request: web.BaseRequest):
         try:
             vc_record = await holder.retrieve_credential_by_id(credential_id)
             await holder.delete_credential(vc_record)
+            topic = "acapy::record::w3c_credential::delete"
+            await session.profile.notify(
+                topic, {"id": credential_id, "state": "deleted"}
+            )
         except StorageNotFoundError as err:
             raise web.HTTPNotFound(reason=err.roll_up) from err
         except StorageError as err:

aries_cloudagent/holder/routes.py

@@ -98,17 +98,15 @@ async def _construct_attr_json(
         if not routing_keys:
             routing_keys = []
 
-        endpoint_dict = {"endpoint": endpoint}
-
         if all_exist_endpoints:
-            all_exist_endpoints[endpoint_type.indy] = endpoint_dict
-            endpoint_dict["routingKeys"] = routing_keys
+            all_exist_endpoints[endpoint_type.indy] = endpoint
+            all_exist_endpoints["routingKeys"] = routing_keys
             attr_json = json.dumps({"endpoint": all_exist_endpoints})
 
         else:
-            endpoint_val = {endpoint_type.indy: endpoint_dict}
+            endpoint_dict = {endpoint_type.indy: endpoint}
             endpoint_dict["routingKeys"] = routing_keys
-            attr_json = json.dumps({"endpoint": endpoint_val})
+            attr_json = json.dumps({"endpoint": endpoint_dict})
 
         return attr_json
 

aries_cloudagent/ledger/base.py

@@ -793,7 +793,7 @@ async def update_endpoint_for_did(
                     )
                     resp = await self._submit(
                         request_json,
-                        True,
+                        sign=True,
                         sign_did=public_info,
                         write_ledger=write_ledger,
                     )
@@ -802,6 +802,7 @@ async def update_endpoint_for_did(
 
             await self._submit(request_json, True, True)
             return True
+
         return False
 
     async def register_nym(

aries_cloudagent/ledger/indy.py

@@ -2308,17 +2308,37 @@ async def test_construct_attr_json_with_routing_keys(self, mock_close, mock_open
             attr_json = await ledger._construct_attr_json(
                 "https://url",
                 EndpointType.ENDPOINT,
-                all_exist_endpoints={"Endpoint": "https://endpoint"},
                 routing_keys=["3YJCx3TqotDWFGv7JMR5erEvrmgu5y4FDqjR7sKWxgXn"],
             )
         assert attr_json == json.dumps(
             {
                 "endpoint": {
-                    "Endpoint": "https://endpoint",
-                    "endpoint": {
-                        "endpoint": "https://url",
-                        "routingKeys": ["3YJCx3TqotDWFGv7JMR5erEvrmgu5y4FDqjR7sKWxgXn"],
-                    },
+                    "endpoint": "https://url",
+                    "routingKeys": ["3YJCx3TqotDWFGv7JMR5erEvrmgu5y4FDqjR7sKWxgXn"],
+                }
+            }
+        )
+
+    @async_mock.patch("aries_cloudagent.ledger.indy.IndySdkLedgerPool.context_open")
+    @async_mock.patch("aries_cloudagent.ledger.indy.IndySdkLedgerPool.context_close")
+    @pytest.mark.asyncio
+    async def test_construct_attr_json_with_routing_keys_all_exist_endpoints(
+        self, mock_close, mock_open
+    ):
+        ledger = IndySdkLedger(IndySdkLedgerPool("name", checked=True), self.profile)
+        async with ledger:
+            attr_json = await ledger._construct_attr_json(
+                "https://url",
+                EndpointType.ENDPOINT,
+                all_exist_endpoints={"profile": "https://endpoint/profile"},
+                routing_keys=["3YJCx3TqotDWFGv7JMR5erEvrmgu5y4FDqjR7sKWxgXn"],
+            )
+        assert attr_json == json.dumps(
+            {
+                "endpoint": {
+                    "profile": "https://endpoint/profile",
+                    "endpoint": "https://url",
+                    "routingKeys": ["3YJCx3TqotDWFGv7JMR5erEvrmgu5y4FDqjR7sKWxgXn"],
                 }
             }
         )