Porównaj commity

...

41 Commity

Autor SHA1 Wiadomość Data
dependabot[bot] a397b71945
Merge 2f824410ff into d6a10b4be0 2024-04-25 14:10:58 +12:00
Ryan Barrett d6a10b4be0
update profile button: make receive task instead of running it inline 2024-04-24 17:06:25 -07:00
Ryan Barrett 41b2aaa1a8
incoming DMs to protocol bot users: filter out @-mentions
for #880
2024-04-24 16:45:43 -07:00
Ryan Barrett 06bf3bf534
receive: drop activity if protocol doesn't own actor
it's probably from a bridged user, and we only want to handle original activities, not bridged ones.

fixes https://console.cloud.google.com/errors/detail/CM6i4sH4176iaQ;time=P30D?project=bridgy-federated
2024-04-24 15:57:00 -07:00
Ryan Barrett 3c62f7cfcc
give up on email for now, still can't get SMTP to work
Gmail SMTP is unhappy with auth, even though I got it to work on https://shell-py3.appspot.com/ , and Gandi SMTP seems to block Google Cloud IPs. sigh.
2024-04-24 15:30:20 -07:00
Ryan Barrett a79dc45b28
email: update from address 2024-04-24 15:07:42 -07:00
Ryan Barrett bfed0452b9
ATProto polls: skip notifs/posts we've already handled 2024-04-24 15:06:49 -07:00
Ryan Barrett 55ae9fd2bb
User.enable/disable_protocol: move email out of datastore tx 2024-04-24 14:34:52 -07:00
Ryan Barrett b543fdb1d5
switch to Gmail SMTP, other minor tweaks 2024-04-24 14:26:20 -07:00
Ryan Barrett def8c3d535
docs: remove link to obsolete error handling section
thanks for the catch @jernst!
2024-04-24 11:42:27 -07:00
Ryan Barrett ece168fac1
email me when someone enables or disables a protocol 2024-04-24 11:15:28 -07:00
dependabot[bot] 139bd0aedb build(deps): bump redis from 5.0.3 to 5.0.4
Bumps [redis](https://github.com/redis/redis-py) from 5.0.3 to 5.0.4.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v5.0.3...v5.0.4)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-24 05:45:28 -07:00
dependabot[bot] f00028e4b9 build(deps): bump websocket-client from 1.7.0 to 1.8.0
Bumps [websocket-client](https://github.com/websocket-client/websocket-client) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/websocket-client/websocket-client/releases)
- [Changelog](https://github.com/websocket-client/websocket-client/blob/master/ChangeLog)
- [Commits](https://github.com/websocket-client/websocket-client/compare/v1.7.0...v1.8.0)

---
updated-dependencies:
- dependency-name: websocket-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-24 12:41:03 +00:00
Ryan Barrett 506de7fd1d
User.get_or_create: if the id is a copy, return the original
...which will be in a different protocol! worried this will break something. hrm.
2024-04-23 17:56:59 -07:00
Ryan Barrett 0f89b0750a
ATProto: remove some obsolete TODOs 2024-04-23 17:56:27 -07:00
Ryan Barrett 4efb0d3e35
ATProto: tweak polling: every 5 min, 10 items each call 2024-04-23 16:58:30 -07:00
Ryan Barrett 0238b46e36
add Protocol.HAS_COPIES, use to generate ids.COPIES_PROTOCOLS 2024-04-23 16:52:53 -07:00
Ryan Barrett c86c91b25b
User.enable_protocols bug fix: only call create_for for copy protocols
also add integration test for following protocol bot user from ATProto
2024-04-23 16:38:08 -07:00
Ryan Barrett 9fe715137a
webfinger: check that AP protocol is enabled 2024-04-23 15:31:24 -07:00
Ryan Barrett ce23a72549
make ids.translate_handle support protocol bot users 2024-04-23 15:01:40 -07:00
Ryan Barrett 0e8b9ece7e
set up ap.brid.gy protocol bot user 2024-04-23 13:54:29 -07:00
Ryan Barrett 11eb082190
tighten common.unwrap so it doesn't remove protocol bot user URLs
...like https://bsky.brid.gy/ . this hopefully fixes following bot users in eg AP to enable protocols.
2024-04-23 12:00:39 -07:00
Ryan Barrett 3f1d860bba
AP: add test for following bot user to enable protocol
also relax Web.put check on id to allow internal domains for bot users
2024-04-23 10:17:13 -07:00
Ryan Barrett f52f7060a6
test_protocol.test_for_id_object_missing_source_protocol: mock network fetch 2024-04-23 10:16:06 -07:00
Ryan Barrett 304994e3b7
Protocol.for_id: return Web for protocol bot users 2024-04-23 09:39:30 -07:00
Ryan Barrett d4a56127d9
incoming webmentions: stop converting home pages to update activities
pass Protocol.receive the bare actor object, let it convert to update activity, like with posts etc
2024-04-23 06:50:55 -07:00
Ryan Barrett 0c41f0e081
remove /webmention-interactive (for old Web-specific update profile button) 2024-04-22 20:25:48 -07:00
Ryan Barrett 115d85909a
abstract "update profile" button across protocols
adds new /[protocol]/[id]/update-profile endpoint
2024-04-22 20:21:56 -07:00
Ryan Barrett 03b0f54cfe
Protocol.receive: wrap bare actor object in update activity 2024-04-22 18:39:27 -07:00
Ryan Barrett 0b00e6eb4b
AP: serve protocol bot users on their own subdomains 2024-04-22 16:07:54 -07:00
Ryan Barrett c87e69d354
bug fix for ids.web_ap_base_domain
...and test it properly instead of mocking out the constant
2024-04-22 15:15:27 -07:00
Ryan Barrett 34692abc60
handle protocol bot users in webfinger, ids.translate_handle, Web.owns_handle
for #880
2024-04-22 14:01:09 -07:00
Ryan Barrett b9551c4de7
adjust Web.owns_id to say it owns protocol bot user domains but not their pages
eg bsky.brid.gy True, https://bsky.brid.gy/ True, https://bsky.brid.gy/foo False.

also move our internal synthetic UI-initiated follow ids from https://fed.brid.gy/web/... to under the user's own domain. hopefully this won't break anything 🤞
2024-04-22 11:58:01 -07:00
Ryan Barrett ed78090d2c
expand User.ap_subdomain to allow protocol subdomains like bsky
part of setting up per-protocol bot users for #880
2024-04-22 11:12:03 -07:00
Ryan Barrett e1f9021696
AP instance actor: move AS2 JSON files, start adding per-protocol bot users 2024-04-21 16:35:17 -07:00
Ryan Barrett 18b1a33d22
testutil.setUp() noop tweak for adding Fake classes to common.*_DOMAINS 2024-04-21 16:22:33 -07:00
Ryan Barrett dcadbccb3a
start to make per-protocol bot users
bsky.brid.gy instance actor, rel-me links
2024-04-21 14:08:26 -07:00
Ryan Barrett 10023d17fd
Protocol.enable_protocol: create copy user if necessary 2024-04-21 12:18:12 -07:00
Ryan Barrett 6b597c90c3
User.get_or_create: abstract propagate and create_for across protocols 2024-04-21 11:40:13 -07:00
Ryan Barrett f357ea1698
ActivityPub: accept non-public DMs to protocol bot users
for #880
2024-04-21 08:36:03 -07:00
dependabot[bot] 2f824410ff
build(deps): bump protobuf from 4.24.3 to 5.26.1
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 4.24.3 to 5.26.1.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v4.24.3...v5.26.1)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 12:10:44 +00:00
34 zmienionych plików z 762 dodań i 243 usunięć

2
.gitignore vendored
Wyświetl plik

@ -10,6 +10,8 @@ flask_secret_key
make_password
private_notes
service_account_creds.json
smtp_password
smtp_user
superfeedr_token
superfeedr_username
TAGS

Wyświetl plik

@ -57,7 +57,9 @@ WEB_OPT_OUT_DOMAINS = None
FEDI_URL_RE = re.compile(r'https://[^/]+/(@|users/)([^/@]+)(@[^/@]+)?(/(?:statuses/)?[0-9]+)?')
_BOT_ACTOR_IDS = None
# can't use translate_user_id because Web.owns_id checks valid_domain, which
# doesn't allow our protocol subdomains
BOT_ACTOR_IDS = tuple(f'https://{domain}/{domain}' for domain in PROTOCOL_DOMAINS)
def instance_actor():
@ -68,15 +70,6 @@ def instance_actor():
return _INSTANCE_ACTOR
def bot_actor_ids():
global _BOT_ACTOR_IDS
if _BOT_ACTOR_IDS is None:
from activitypub import ActivityPub
_BOT_ACTOR_IDS = [translate_user_id(id=domain, from_=Web, to=ActivityPub)
for domain in PROTOCOL_DOMAINS]
return _BOT_ACTOR_IDS
class ActivityPub(User, Protocol):
"""ActivityPub protocol class.
@ -842,7 +835,12 @@ def postprocess_as2_actor(actor, user):
@flask_util.cached(cache, CACHE_TIME)
def actor(handle_or_id):
"""Serves a user's AS2 actor from the datastore."""
cls = Protocol.for_request(fed='web')
if handle_or_id == PRIMARY_DOMAIN or handle_or_id in PROTOCOL_DOMAINS:
from web import Web
cls = Web
else:
cls = Protocol.for_request(fed='web')
if not cls:
error(f"Couldn't determine protocol", status=404)
elif cls.LABEL == 'web' and request.path.startswith('/ap/'):
@ -939,7 +937,12 @@ def inbox(protocol=None, id=None):
# those as explicitly public. Use as2's is_public instead of as1's because
# as1's interprets unlisted as true.
# TODO: move this to Protocol
if type == 'Create' and not as2.is_public(activity, unlisted=False):
object = as1.get_object(activity)
to_cc = set(as1.get_ids(object, 'to') + as1.get_ids(activity, 'cc') +
as1.get_ids(object, 'to') + as1.get_ids(object, 'cc'))
if (type == 'Create' and not as2.is_public(activity, unlisted=False)
# DM to one of our protocol bot users
and not (len(to_cc) == 1 and to_cc.pop() in BOT_ACTOR_IDS)):
logger.info('Dropping non-public activity')
return 'OK'

Wyświetl plik

@ -0,0 +1,33 @@
{
"@context": [
"https://www.w3.org/ns/activitystreams",
"https://w3id.org/security/v1"
],
"type": "Application",
"id": "https://ap.brid.gy/ap.brid.gy",
"url": "https://ap.brid.gy/",
"preferredUsername": "ap.brid.gy",
"summary": "<a href='https://fed.brid.gy/'>Bridgy Fed</a> bot user for the <a href='https://en.wikipedia.org/wiki/Fediverse'>fediverse</a>. To bridge your Bluesky account to the fediverse, follow this account. <a href='https://fed.brid.gy/docs'>More info here.</a>",
"name": "Bridgy Fed for the fediverse",
"attachment": [{
"name": "Web site",
"type": "PropertyValue",
"value": "<a rel=\"me\" href=\"https://fed.brid.gy\"><span class=\"invisible\">https://</span>fed.brid.gy</a>"
}],
"image": [
{
"type": "Image",
"url": "https://fed.brid.gy/static/bridgy_logo.jpg"
},
{
"name": "Bridgy Fed for the fediverse",
"type": "Image",
"url": "https://fed.brid.gy/static/bridgy_logo_square.jpg"
}
],
"icon": {
"name": "Bridgy Fed for the fediverse",
"type": "Image",
"url": "https://fed.brid.gy/static/bridgy_logo_square.jpg"
}
}

Wyświetl plik

@ -86,16 +86,14 @@ class ATProto(User, Protocol):
https://atproto.com/specs/did
"""
ABBREV = 'bsky'
# TODO: add second bsky label? inject into PROTOCOLS?
PHRASE = 'Bluesky'
LOGO_HTML = '<img src="/oauth_dropins_static/bluesky.svg">'
# note that PDS hostname is atproto.brid.gy here, not bsky.brid.gy. Bluesky
# team currently has our hostname as atproto.brid.gy in their federation
# test.
# TODO: switch to bsky.brid.gy once they lift their federation limits? we'd
# need to update serviceEndpoint in all users' DID docs. :/
PDS_URL = f'https://atproto{common.SUPERDOMAIN}/'
CONTENT_TYPE = 'application/json'
HAS_COPIES = True
DEFAULT_ENABLED_PROTOCOLS = ()
def _pre_put_hook(self):
@ -237,7 +235,7 @@ class ATProto(User, Protocol):
return None
def is_blocklisted(url):
def is_blocklisted(url, allow_internal=False):
# don't block common.DOMAINS since we want ourselves, ie our own PDS, to
# be a valid domain to send to
return util.domain_or_parent_in(util.domain_from_link(url), DOMAIN_BLOCKLIST)
@ -444,9 +442,6 @@ class ATProto(User, Protocol):
Returns:
bool: True if the object was fetched and populated successfully,
False otherwise
Raises:
TODO
"""
id = obj.key.id()
if not cls.owns_id(id):
@ -567,7 +562,7 @@ def poll_notifications():
repos = {r.key.id(): r for r in AtpRepo.query()}
logger.info(f'Got {len(repos)} repos')
if not repos:
return
return 'Nothing to do ¯\_(ツ)_/¯', 204
users = itertools.chain(*(cls.query(cls.copies.uri.IN(list(repos)))
for cls in set(PROTOCOLS.values())
@ -593,10 +588,9 @@ def poll_notifications():
client.session['accessJwt'] = service_jwt(os.environ['APPVIEW_HOST'],
repo_did=did,
privkey=repo.signing_key)
resp = client.app.bsky.notification.listNotifications()
resp = client.app.bsky.notification.listNotifications(limit=10)
for notif in resp['notifications']:
actor_did = notif['author']['did']
logger.debug(f'Got {notif["reason"]} from {notif["author"]["handle"]} {notif["uri"]} {notif["cid"]} : {json_dumps(notif, indent=2)}')
# TODO: verify sig. skipping this for now because we're getting
# these from the AppView, which is trusted, specifically we expect
@ -604,6 +598,11 @@ def poll_notifications():
obj = Object.get_or_create(id=notif['uri'], bsky=notif['record'],
source_protocol=ATProto.ABBREV,
actor=actor_did)
if obj.status in ('complete', 'ignored'):
continue
logger.debug(f'Got new {notif["reason"]} from {notif["author"]["handle"]} {notif["uri"]} {notif["cid"]} : {json_dumps(notif, indent=2)}')
if not obj.status:
obj.status = 'new'
obj.add('notify', user.key)
@ -626,7 +625,7 @@ def poll_posts():
repos = {r.key.id(): r for r in AtpRepo.query()}
logger.info(f'Got {len(repos)} repos')
if not repos:
return
return 'Nothing to do ¯\_(ツ)_/¯', 204
users = itertools.chain(*(cls.query(cls.copies.uri.IN(list(repos)))
for cls in set(PROTOCOLS.values())
@ -652,7 +651,7 @@ def poll_posts():
client.session['accessJwt'] = service_jwt(os.environ['APPVIEW_HOST'],
repo_did=did,
privkey=repo.signing_key)
resp = client.app.bsky.feed.getTimeline()
resp = client.app.bsky.feed.getTimeline(limit=10)
for item in resp['feed']:
# TODO: handle reposts once we have a URI for them
# https://github.com/bluesky-social/atproto/issues/1811
@ -660,7 +659,6 @@ def poll_posts():
continue
post = item['post']
logger.debug(f'Got {post["uri"]}: {json_dumps(item, indent=2)}')
# TODO: verify sig. skipping this for now because we're getting
# these from the AppView, which is trusted, specifically we expect
@ -669,6 +667,10 @@ def poll_posts():
obj = Object.get_or_create(id=post['uri'], bsky=post['record'],
source_protocol=ATProto.ABBREV,
actor=author_did)
if obj.status in ('complete', 'ignored'):
continue
logger.debug(f'Got new post: {post["uri"]} : {json_dumps(item, indent=2)}')
if not obj.status:
obj.status = 'new'
obj.add('feed', user.key)

Wyświetl plik

@ -0,0 +1,33 @@
{
"@context": [
"https://www.w3.org/ns/activitystreams",
"https://w3id.org/security/v1"
],
"type": "Application",
"id": "https://bsky.brid.gy/bsky.brid.gy",
"url": "https://bsky.brid.gy/",
"preferredUsername": "bsky.brid.gy",
"summary": "<a href='https://fed.brid.gy/'>Bridgy Fed</a> bot user for <a href='https://bsky.social/'>Bluesky</a>. To bridge your fediverse account to Bluesky, follow this account or reply <em>yes</em> when it promps you with a DM. <a href='https://fed.brid.gy/docs'>More info here.</a>",
"name": "Bridgy Fed for Bluesky",
"attachment": [{
"name": "Web site",
"type": "PropertyValue",
"value": "<a rel=\"me\" href=\"https://fed.brid.gy\"><span class=\"invisible\">https://</span>fed.brid.gy</a>"
}],
"image": [
{
"type": "Image",
"url": "https://fed.brid.gy/static/bridgy_logo.jpg"
},
{
"name": "Bridgy Fed for Bluesky",
"type": "Image",
"url": "https://fed.brid.gy/static/bridgy_logo_square.jpg"
}
],
"icon": {
"name": "Bridgy Fed for Bluesky",
"type": "Image",
"url": "https://fed.brid.gy/static/bridgy_logo_square.jpg"
}
}

Wyświetl plik

@ -38,13 +38,12 @@ SUPERDOMAIN = '.brid.gy'
# use it to canonicalize most UI routes from these to fed.brid.gy.
PROTOCOL_DOMAINS = (
'ap.brid.gy',
'atp.brid.gy',
'atproto.brid.gy',
'bluesky.brid.gy',
'bsky.brid.gy',
'fa.brid.gy',
'nostr.brid.gy',
'web.brid.gy',
'eefake.brid.gy',
'fa.brid.gy',
'other.brid.gy',
)
OTHER_DOMAINS = (
'bridgy-federated.appspot.com',
@ -64,6 +63,9 @@ DOMAIN_BLOCKLIST = (
'twitter.com',
)
SMTP_HOST = 'smtp.gmail.com'
SMTP_PORT = 587
# populated in models.reset_protocol_properties
SUBDOMAIN_BASE_URL_RE = None
ID_FIELDS = ['id', 'object', 'actor', 'author', 'inReplyTo', 'url']
@ -200,10 +202,11 @@ def unwrap(val, field=None):
return [unwrap(v) for v in val]
elif isinstance(val, str):
unwrapped = SUBDOMAIN_BASE_URL_RE.sub('', val)
if field in ID_FIELDS and re.fullmatch(DOMAIN_RE, unwrapped):
unwrapped = f'https://{unwrapped}/'
return unwrapped
if match := SUBDOMAIN_BASE_URL_RE.match(val):
unwrapped = match.group('path')
if field in ID_FIELDS and re.fullmatch(DOMAIN_RE, unwrapped):
return f'https://{unwrapped}/'
return unwrapped
return val
@ -305,3 +308,11 @@ def create_task(queue, delay=None, **params):
msg = f'Added {queue} task {task.name} : {params}'
logger.info(msg)
return msg, 202
def email_me(msg):
assert False # not working, SMTP woes :(
if not DEBUG:
util.send_email(smtp_host=SMTP_HOST, smtp_port=SMTP_PORT,
from_='scufflechuck@gmail.com', to='bridgy-fed@ryanb.org',
subject=util.ellipsize(msg), body=msg)

Wyświetl plik

@ -4,10 +4,10 @@
cron:
- description: ATProto poll posts
url: /queue/atproto-poll-posts
schedule: every 15 minutes
schedule: every 5 minutes
target: hub
- description: ATProto poll notifications
url: /queue/atproto-poll-notifs
schedule: every 15 minutes
schedule: every 5 minutes
target: hub

Wyświetl plik

@ -0,0 +1,8 @@
{
"type": "Application",
"id": "https://eefake.brid.gy/eefake.brid.gy",
"url": "https://eefake.brid.gy/",
"preferredUsername": "eefake.brid.gy",
"summary": "Only for unit tests",
"name": "ExplicitEnableFake protocol class in testutil"
}

Wyświetl plik

@ -0,0 +1,8 @@
{
"type": "Application",
"id": "https://fake.brid.gy/fake.brid.gy",
"url": "https://fake.brid.gy/",
"preferredUsername": "fake.brid.gy",
"summary": "Only for unit tests",
"name": "Fake protocol class in testutil"
}

Wyświetl plik

@ -122,7 +122,7 @@ class FollowCallback(indieauth.Callback):
followee_id = followee.as1.get('id')
timestamp = util.now().replace(microsecond=0, tzinfo=None).isoformat()
follow_id = common.host_url(user.user_page_path(f'following#{timestamp}-{addr}'))
follow_id = f'{user.web_url()}#follow-{timestamp}-{addr}'
follow_as1 = {
'objectType': 'activity',
'verb': 'follow',
@ -204,7 +204,7 @@ class UnfollowCallback(indieauth.Callback):
# TODO(#529): generalize
timestamp = util.now().replace(microsecond=0, tzinfo=None).isoformat()
unfollow_id = common.host_url(user.user_page_path(f'following#undo-{timestamp}-{followee_id}'))
unfollow_id = f'{user.web_url()}#unfollow-{timestamp}-{followee_id}'
unfollow_as1 = {
'objectType': 'activity',
'verb': 'stop-following',

42
ids.py
Wyświetl plik

@ -11,17 +11,26 @@ from google.cloud.ndb.query import FilterNode, Query
from granary.bluesky import BSKY_APP_URL_RE, web_url_to_at_uri
from oauth_dropins.webutil import util
from common import subdomain_wrap, LOCAL_DOMAINS, PRIMARY_DOMAIN, SUPERDOMAIN
from common import (
LOCAL_DOMAINS,
PRIMARY_DOMAIN,
PROTOCOL_DOMAINS,
subdomain_wrap,
SUPERDOMAIN,
)
import models
logger = logging.getLogger(__name__)
# Protocols to check User.copies and Object.copies before translating
COPIES_PROTOCOLS = ('atproto', 'fake', 'other', 'nostr')
# populated in models.reset_protocol_properties
COPIES_PROTOCOLS = None
# Web user domains whose AP actor ids are on fed.brid.gy, not web.brid.gy, for
# historical compatibility. Loaded on first call to web_ap_subdomain().
_FED_SUBDOMAIN_SITES = None
#
# Maps string domain to string subdomain (bsky, fed, or web).
_NON_WEB_SUBDOMAIN_SITES = None
def web_ap_base_domain(user_domain):
@ -40,16 +49,18 @@ def web_ap_base_domain(user_domain):
if request.host in LOCAL_DOMAINS:
return request.host_url
global _FED_SUBDOMAIN_SITES
if _FED_SUBDOMAIN_SITES is None:
_FED_SUBDOMAIN_SITES = {
key.id() for key in Query('MagicKey',
filters=FilterNode('ap_subdomain', '=', 'fed')
).fetch(keys_only=True)
global _NON_WEB_SUBDOMAIN_SITES
if _NON_WEB_SUBDOMAIN_SITES is None:
_NON_WEB_SUBDOMAIN_SITES = {
user.key.id(): user.ap_subdomain
for user in Query('MagicKey',
filters=FilterNode('ap_subdomain', '!=', 'web'),
projection=['ap_subdomain'],
).fetch()
}
logger.info(f'Loaded {len(_FED_SUBDOMAIN_SITES)} fed subdomain Web users')
logger.info(f'Loaded {len(_NON_WEB_SUBDOMAIN_SITES)} non-web.brid.gy Web users')
subdomain = 'fed' if user_domain in _FED_SUBDOMAIN_SITES else 'web'
subdomain = _NON_WEB_SUBDOMAIN_SITES.get(user_domain, 'web')
return f'https://{subdomain}{SUPERDOMAIN}/'
@ -149,13 +160,16 @@ def translate_handle(*, handle, from_, to, enhanced):
match from_.LABEL, to.LABEL:
case _, 'activitypub':
domain = handle if enhanced else f'{from_.ABBREV}{SUPERDOMAIN}'
domain = f'{from_.ABBREV}{SUPERDOMAIN}'
if enhanced or handle == PRIMARY_DOMAIN or handle in PROTOCOL_DOMAINS:
domain = handle
return f'@{handle}@{domain}'
case _, 'atproto' | 'nostr':
handle = handle.lstrip('@').replace('@', '.')
return (handle if enhanced
else f'{handle}.{from_.ABBREV}{SUPERDOMAIN}')
if enhanced or handle == PRIMARY_DOMAIN or handle in PROTOCOL_DOMAINS:
return handle
return f'{handle}.{from_.ABBREV}{SUPERDOMAIN}'
case 'activitypub', 'web':
user, instance = handle.lstrip('@').split('@')

Wyświetl plik

@ -119,7 +119,9 @@ def reset_protocol_properties():
abbrevs = f'({"|".join(PROTOCOLS.keys())}|fed)'
common.SUBDOMAIN_BASE_URL_RE = re.compile(
rf'^https?://({abbrevs}\.brid\.gy|localhost(:8080)?)/(convert/|r/)?({abbrevs}/)?')
rf'^https?://({abbrevs}\.brid\.gy|localhost(:8080)?)/(convert/|r/)?({abbrevs}/)?(?P<path>.+)')
ids.COPIES_PROTOCOLS = tuple(label for label, proto in PROTOCOLS.items()
if proto and proto.HAS_COPIES)
class User(StringIdModel, metaclass=ProtocolUserMeta):
@ -235,13 +237,15 @@ class User(StringIdModel, metaclass=ProtocolUserMeta):
old_val = getattr(user, field, None)
new_val = kwargs.get(field)
if ((old_val is None and new_val is not None)
or (field == 'direct' and not old_val and new_val)):
or (field == 'direct' and not old_val and new_val)):
setattr(user, field, new_val)
user.put()
if not propagate:
return user
else:
if orig := get_original(id):
return orig
user = cls(id=id, **kwargs)
user.existing = False
@ -249,12 +253,14 @@ class User(StringIdModel, metaclass=ProtocolUserMeta):
if not user.obj_key:
user.obj = cls.load(user.profile_id())
ATProto = PROTOCOLS['atproto']
if propagate and cls.LABEL != 'atproto' and not user.get_copy(ATProto):
if cls.is_enabled_to(ATProto, user=id):
ATProto.create_for(user)
else:
logger.info(f'{cls.LABEL} <=> atproto not enabled, skipping')
if propagate:
for label in ids.COPIES_PROTOCOLS:
proto = PROTOCOLS[label]
if proto != cls and not user.get_copy(proto):
if cls.is_enabled_to(proto, user=id):
proto.create_for(user)
else:
logger.info(f'{cls.LABEL} <=> atproto not enabled, skipping')
# generate keys for all protocols _except_ our own
#
@ -358,25 +364,40 @@ class User(StringIdModel, metaclass=ProtocolUserMeta):
Args:
to_proto (:class:`protocol.Protocol` subclass)
"""
user = self.key.get()
add(user.enabled_protocols, to_proto.LABEL)
user.put()
@ndb.transactional()
def enable():
user = self.key.get()
add(user.enabled_protocols, to_proto.LABEL)
if to_proto.LABEL in ids.COPIES_PROTOCOLS and not user.get_copy(to_proto):
to_proto.create_for(user)
user.put()
enable()
add(self.enabled_protocols, to_proto.LABEL)
@ndb.transactional()
msg = f'Enabled {to_proto.LABEL} for {self.key.id()} : {self.user_page_path()}'
logger.info(msg)
def disable_protocol(self, to_proto):
"""Removes ``to_proto` from :attr:`enabled_protocols`.
Args:
to_proto (:class:`protocol.Protocol` subclass)
"""
user = self.key.get()
remove(user.enabled_protocols, to_proto.LABEL)
user.put()
@ndb.transactional()
def disable():
user = self.key.get()
remove(user.enabled_protocols, to_proto.LABEL)
# TODO: delete copy user
# https://github.com/snarfed/bridgy-fed/issues/783
user.put()
disable()
remove(self.enabled_protocols, to_proto.LABEL)
msg = f'Disabled {to_proto.LABEL} for {self.key.id()} : {self.user_page_path()}'
logger.info(msg)
def handle_as(self, to_proto):
"""Returns this user's handle in a different protocol.
@ -521,7 +542,8 @@ class User(StringIdModel, metaclass=ProtocolUserMeta):
Returns:
str:
"""
if isinstance(self, proto):
# don't use isinstance because the testutil Fake protocol has subclasses
if self.LABEL == proto.LABEL:
return self.key.id()
for copy in self.copies:

Wyświetl plik

@ -0,0 +1,8 @@
{
"type": "Application",
"id": "https://other.brid.gy/other.brid.gy",
"url": "https://other.brid.gy/",
"preferredUsername": "other.brid.gy",
"summary": "Only for unit tests",
"name": "OtherFake protocol class in testutil"
}

Wyświetl plik

@ -19,6 +19,8 @@ from oauth_dropins.webutil.flask_util import (
flash,
redirect,
)
import requests
import werkzeug.exceptions
import common
from common import DOMAIN_RE
@ -27,6 +29,7 @@ import ids
from models import fetch_objects, fetch_page, Follower, Object, PAGE_SIZE, PROTOCOLS
from protocol import Protocol
# precompute this because we get a ton of requests for non-existing users
# from weird open redirect referrers:
# https://github.com/snarfed/bridgy-fed/issues/422
@ -160,6 +163,27 @@ def notifications(protocol, id):
return render_template('notifications.html', **TEMPLATE_VARS, **locals())
@app.post(f'/<any({",".join(PROTOCOLS)}):protocol>/<id>/update-profile')
@canonicalize_request_domain(common.PROTOCOL_DOMAINS, common.PRIMARY_DOMAIN)
def update_profile(protocol, id):
user = load_user(protocol, id)
try:
profile_obj = user.load(user.profile_id(), remote=True)
except (requests.RequestException, werkzeug.exceptions.HTTPException) as e:
_, msg = util.interpret_http_exception(e)
flash(f"Couldn't update profile for {user.handle_or_id()}: {msg}")
if profile_obj:
common.create_task(queue='receive', obj=profile_obj.key.urlsafe(),
authed_as=id)
flash(f'Updating profile for {user.handle_or_id()}')
else:
flash(f"Couldn't update profile for {user.handle_or_id()}")
return redirect(user.user_page_path(), code=302)
@app.get(f'/<any({",".join(PROTOCOLS)}):protocol>/<id>/<any(followers,following):collection>')
@canonicalize_request_domain(common.PROTOCOL_DOMAINS, common.PRIMARY_DOMAIN)
def followers_or_following(protocol, id, collection):

Wyświetl plik

@ -4,14 +4,14 @@ from datetime import timedelta
import logging
import re
from threading import Lock
from urllib.parse import urljoin
from urllib.parse import urljoin, urlparse
from cachetools import cached, LRUCache
from flask import g, request
from google.cloud import ndb
from google.cloud.ndb import OR
from google.cloud.ndb.model import _entity_to_protobuf
from granary import as1
from granary import as1, as2
from oauth_dropins.webutil.appengine_info import DEBUG
from oauth_dropins.webutil.flask_util import cloud_tasks_only
from oauth_dropins.webutil import models
@ -80,6 +80,9 @@ class Protocol:
appropriate for the ``Content-Type`` HTTP header.
HAS_FOLLOW_ACCEPTS (bool): whether this protocol supports explicit
accept/reject activities in response to follows, eg ActivityPub
HAS_COPIES (bool): whether this protocol is push and needs us to
proactively create "copy" users and objects, as opposed to pulling
converted objects on demand
DEFAULT_ENABLED_PROTOCOLS (list of str): labels of other protocols that
are automatically enabled for this protocol to bridge into
"""
@ -89,6 +92,7 @@ class Protocol:
LOGO_HTML = ''
CONTENT_TYPE = None
HAS_FOLLOW_ACCEPTS = False
HAS_COPIES = False
DEFAULT_ENABLED_PROTOCOLS = ()
def __init__(self):
@ -139,6 +143,7 @@ class Protocol:
label = domain.removesuffix(common.SUPERDOMAIN)
return PROTOCOLS.get(label)
# TODO: redesign this API to require user
@classmethod
def is_enabled_to(from_cls, to_cls, user=None):
"""Returns True if two protocols, and optionally a user, can be bridged.
@ -301,10 +306,11 @@ class Protocol:
if not id:
return None
# step 1: check for our per-protocol subdomains
if util.is_web(id):
# step 1: check for our per-protocol subdomains
is_homepage = urlparse(id).path.strip('/') == ''
by_subdomain = Protocol.for_bridgy_subdomain(id)
if by_subdomain:
if by_subdomain and not is_homepage:
logger.info(f' {by_subdomain.LABEL} owns id {id}')
return by_subdomain
@ -441,6 +447,18 @@ class Protocol:
if owner:
return cls.key_for(owner)
@classmethod
def create_for(cls, user):
"""Creates a copy user in this protocol.
Should add the copy user to :attr:`copies`.
Args:
user (models.User): original source user. Shouldn't already have a
copy user for this protocol in :attr:`copies`.
"""
raise NotImplementedError()
@classmethod
def send(to_cls, obj, url, from_user=None, orig_obj=None):
"""Sends an outgoing activity.
@ -535,18 +553,22 @@ class Protocol:
raise NotImplementedError()
@classmethod
def is_blocklisted(cls, url):
def is_blocklisted(cls, url, allow_internal=False):
"""Returns True if we block the given URL and shouldn't deliver to it.
Default implementation here, subclasses may override.
Args:
url (str):
allow_internal (bool): whether to return False for internal domains
like ``fed.brid.gy``, ``bsky.brid.gy``, etc
Returns: bool:
"""
return util.domain_or_parent_in(util.domain_from_link(url),
DOMAIN_BLOCKLIST + DOMAINS)
blocklist = DOMAIN_BLOCKLIST
if not allow_internal:
blocklist += DOMAINS
return util.domain_or_parent_in(util.domain_from_link(url), blocklist)
@classmethod
def translate_ids(to_cls, obj):
@ -662,7 +684,7 @@ class Protocol:
if not id:
error('No id provided')
elif from_cls.is_blocklisted(id) and not internal:
elif from_cls.is_blocklisted(id, allow_internal=internal):
error(f'Activity {id} is blocklisted')
# short circuit if we've already seen this activity id.
@ -685,6 +707,8 @@ class Protocol:
actor = as1.get_owner(obj.as1)
if not actor:
error('Activity missing actor or author', status=400)
elif from_cls.owns_id(actor) is False:
error(f"{from_cls.LABEL} doesn't own actor {actor}, this is probably a bridged activity. Skipping.", status=204)
if authed_as:
assert isinstance(authed_as, str)
@ -805,18 +829,22 @@ class Protocol:
return 'OK', 200
elif obj.type == 'post':
to_cc = (util.get_list(inner_obj_as1, 'to')
+ util.get_list(inner_obj_as1, 'cc'))
if len(to_cc) == 1 and to_cc[0] in PROTOCOL_DOMAINS:
content = inner_obj_as1.get('content').strip().lower()
logger.info(f'DM to bot user {to_cc}: {content}')
to_cc = (as1.get_ids(inner_obj_as1, 'to')
+ as1.get_ids(inner_obj_as1, 'cc'))
if len(to_cc) == 1 and to_cc != [as2.PUBLIC_AUDIENCE]:
proto = Protocol.for_bridgy_subdomain(to_cc[0])
assert proto
if content in ('yes', 'ok'):
from_user.enable_protocol(proto)
elif content == 'no':
from_user.disable_protocol(proto)
return 'OK', 200
if proto:
# remove @-mentions of bot user in HTML links
soup = util.parse_html(inner_obj_as1.get('content', ''))
for link in soup.find_all('a'):
link.extract()
content = soup.get_text().strip().lower()
logger.info(f'got DM to {to_cc}: {content}')
if content in ('yes', 'ok'):
from_user.enable_protocol(proto)
elif content == 'no':
from_user.disable_protocol(proto)
return 'OK', 200
# fetch actor if necessary
if actor and actor.keys() == set(['id']):
@ -973,7 +1001,7 @@ class Protocol:
Returns:
models.Object: ``obj`` if it's an activity, otherwise a new object
"""
if obj.type not in ('note', 'article', 'comment'):
if obj.type not in set(('note', 'article', 'comment')) | as1.ACTOR_TYPES:
return obj
obj_actor = as1.get_owner(obj.as1)
@ -1300,8 +1328,8 @@ class Protocol:
if obj.source_protocol:
logger.warning(f'Object {obj.key.id()} changed protocol from {obj.source_protocol} to {cls.LABEL} ?!')
obj.source_protocol = cls.LABEL
obj.put()
obj.put()
with objects_cache_lock:
objects_cache[id] = obj
return obj

Wyświetl plik

@ -74,7 +74,7 @@ pkce==1.0.3
praw==7.7.1
prawcore==2.4.0
proto-plus==1.23.0
protobuf==4.24.3
protobuf==5.26.1
pyasn1==0.6.0
pyasn1-modules==0.4.0
pycparser==2.22
@ -87,7 +87,7 @@ python-dateutil==2.9.0.post0
python-tumblpy==1.1.4
pytz==2024.1
PyYAML==6.0.1
redis==5.0.3
redis==5.0.4
requests==2.31.0
requests-oauthlib==1.4.0
rsa==4.9
@ -105,7 +105,7 @@ update-checker==0.18.0
urllib3==2.2.1
webencodings==0.5.1
WebOb==1.8.7
websocket-client==1.7.0
websocket-client==1.8.0
websockets==12.0
Werkzeug==3.0.2
wrapt==1.16.0

Wyświetl plik

@ -104,7 +104,7 @@
<li><a href="#compare">How do the different protocols compare?</a></li>
<li><a href="#translate">How are the different protocols translated?</a></li>
<li><a href="#router">How are activities routed?</a></li>
<li><a href="#error-handling">How are errors handled?</a></li>
<!-- <li><a href="#error-handling">How are errors handled?</a></li> -->
</ul>

Wyświetl plik

@ -9,7 +9,10 @@
<!-- For Mastodon profile link verification
https://fed.brid.gy/docs#mastodon-link-verification
-->
<a rel="me" href="https://ap.brid.gy/"></a>
<a rel="me" href="https://bsky.brid.gy/"></a>
<a rel="me" href="https://fed.brid.gy/"></a>
<a rel="me" href="https://web.brid.gy/"></a>
<div id="front-form" class="row front-dark">
<div id="topology" style="position: absolute; top: 0; left: 0"></div>

Wyświetl plik

@ -47,13 +47,10 @@
{{ user.handle_or_id() }}
</a>
{% if user.LABEL == 'web' %}
<form method="post" action="/webmention-interactive">
<input name="source" type="hidden" value="{{ user.web_url() }}" />
<button id="update-profile-button" type="submit" title="Update profile"
class="btn btn-default glyphicon glyphicon-refresh"></button>
</form>
{% endif %}
<form method="post" action="{{ user.user_page_path('update-profile') }}">
<button id="update-profile-button" type="submit" title="Update profile"
class="btn btn-default glyphicon glyphicon-refresh"></button>
</form>
</nobr>
</span>

Wyświetl plik

@ -11,16 +11,17 @@ from flask import g
from google.cloud import ndb
from granary import as2, microformats2
from httpsig import HeaderSigner
from oauth_dropins.webutil.flask_util import NoContent
from oauth_dropins.webutil.testutil import requests_response
from oauth_dropins.webutil.util import domain_from_link, json_dumps, json_loads
from oauth_dropins.webutil import util
from oauth_dropins.webutil.util import domain_from_link, json_dumps, json_loads
import requests
from urllib3.exceptions import ReadTimeoutError
from werkzeug.exceptions import BadGateway, BadRequest
# import first so that Fake is defined before URL routes are registered
from . import testutil
from .testutil import Fake, TestCase
from .testutil import ExplicitEnableFake, Fake, TestCase
import activitypub
from activitypub import (
@ -490,13 +491,31 @@ class ActivityPubTest(TestCase):
got = self.client.get('/user.com')
self.assertEqual(404, got.status_code)
# skip _pre_put_hook since it doesn't allow internal domains
@patch.object(Web, '_pre_put_hook', new=lambda self: None)
def test_actor_protocol_bot_user(self, *_):
"""Web users are special cased to drop the /web/ prefix."""
actor_as2 = json_loads(util.read('bsky.brid.gy.as2.json'))
self.make_user('bsky.brid.gy', cls=Web, ap_subdomain='bsky',
obj_as2=actor_as2, obj_id='https://bsky.brid.gy/')
got = self.client.get('/bsky.brid.gy', base_url='https://bsky.brid.gy/')
self.assertEqual(200, got.status_code)
self.assertEqual(as2.CONTENT_TYPE_LD_PROFILE, got.headers['Content-Type'])
self.assert_equals(actor_as2, got.json,
ignore=['inbox', 'outbox', 'endpoints', 'followers',
'following', 'publicKey', 'publicKeyPem'])
# skip _pre_put_hook since it doesn't allow internal domains
@patch.object(Web, '_pre_put_hook', new=lambda self: None)
def test_instance_actor_fetch(self, *_):
def reset_instance_actor():
activitypub._INSTANCE_ACTOR = testutil.global_user
self.addCleanup(reset_instance_actor)
actor_as2 = json_loads(util.read('static/instance-actor.as2.json'))
self.make_user(common.PRIMARY_DOMAIN, cls=Web, obj_as2=actor_as2)
actor_as2 = json_loads(util.read('fed.brid.gy.as2.json'))
self.make_user(common.PRIMARY_DOMAIN, cls=Web, obj_as2=actor_as2,
obj_id='https://fed.brid.gy/')
activitypub._INSTANCE_ACTOR = None
got = self.client.get(f'/{common.PRIMARY_DOMAIN}')
@ -847,6 +866,45 @@ class ActivityPubTest(TestCase):
self.assertIsNone(Object.get_by_id(not_public['id']))
self.assertIsNone(Object.get_by_id(not_public['object']['id']))
def test_follow_bot_user_enables_protocol(self, *mocks):
user = self.make_user('https://mas.to/users/swentel', cls=ActivityPub,
obj_as2=ACTOR)
self.assertFalse(ActivityPub.is_enabled_to(ExplicitEnableFake, user))
id = 'https://inst/follow'
with self.assertRaises(NoContent):
ActivityPub.receive(Object(id=id, as2={
'type': 'Follow',
'id': id,
'actor': 'https://mas.to/users/swentel',
'object': 'https://eefake.brid.gy/eefake.brid.gy',
}))
self.assertEqual(['https://mas.to/users/swentel'],
ExplicitEnableFake.created_for)
user = user.key.get()
self.assertTrue(ActivityPub.is_enabled_to(ExplicitEnableFake, user))
def test_inbox_dm_yes_to_bot_user_enables_protocol(self, *mocks):
user = self.make_user(ACTOR['id'], cls=ActivityPub)
self.assertFalse(ActivityPub.is_enabled_to(ExplicitEnableFake, user))
got = self.post('/ap/sharedInbox', json={
'type': 'Create',
'id': 'https://mas.to/dm#create',
'to': ['https://eefake.brid.gy/eefake.brid.gy'],
'object': {
'type': 'Note',
'id': 'https://mas.to/dm',
'attributedTo': ACTOR['id'],
'to': ['https://eefake.brid.gy/eefake.brid.gy'],
'content': 'yes',
},
})
self.assertEqual(200, got.status_code, got.get_data(as_text=True))
user = user.key.get()
self.assertTrue(ActivityPub.is_enabled_to(ExplicitEnableFake, user))
def test_inbox_actor_blocklisted(self, mock_head, mock_get, mock_post):
got = self.post('/ap/sharedInbox', json={
'type': 'Delete',
@ -904,11 +962,12 @@ class ActivityPubTest(TestCase):
self.assert_user(ActivityPub, 'https://mas.to/actor', obj_as2=LIKE_ACTOR)
def test_inbox_like_no_object_error(self, *_):
Fake.fetchable = {'fake:user': {'id': 'fake:user'}}
swentel = self.make_user('https://inst/user', cls=ActivityPub)
got = self.post('/inbox', json={
'id': 'fake:like',
'id': 'https://inst/like',
'type': 'Like',
'actor': 'fake:user',
'actor': 'https://inst/user',
'object': None,
})
self.assertEqual(400, got.status_code)
@ -1805,6 +1864,7 @@ class ActivityPubUtilsTest(TestCase):
self.assertFalse(ActivityPub.owns_id('https://twitter.com/foo'))
self.assertFalse(ActivityPub.owns_id('https://fed.brid.gy/foo'))
self.assertFalse(ActivityPub.owns_id('https://ap.brid.gy/foo'))
def test_owns_handle(self):
for handle in ('@user@instance', 'user@instance.com', 'user.com@instance.com',

Wyświetl plik

@ -60,6 +60,7 @@ NOTE_AS = {
}
@patch('ids.COPIES_PROTOCOLS', ['atproto'])
class ATProtoTest(TestCase):
def setUp(self):
@ -1191,7 +1192,7 @@ class ATProtoTest(TestCase):
self.assertEqual(200, resp.status_code)
expected_list_notifs = call(
'https://api.bsky-sandbox.dev/xrpc/app.bsky.notification.listNotifications',
'https://api.bsky-sandbox.dev/xrpc/app.bsky.notification.listNotifications?limit=10',
json=None, data=None,
headers={
'Content-Type': 'application/json',
@ -1289,7 +1290,7 @@ class ATProtoTest(TestCase):
self.assertEqual(200, resp.status_code)
get_timeline = call(
'https://api.bsky-sandbox.dev/xrpc/app.bsky.feed.getTimeline',
'https://api.bsky-sandbox.dev/xrpc/app.bsky.feed.getTimeline?limit=10',
json=None, data=None,
headers={
'Content-Type': 'application/json',

Wyświetl plik

@ -62,9 +62,12 @@ class CommonTest(TestCase):
def test_unwrap_protocol_subdomain(self):
for input, expected in [
('https://fa.brid.gy/', ''),
('https://fa.brid.gy/ap/fake:foo', 'fake:foo'),
('https://bsky.brid.gy/convert/ap/did:plc:123', 'did:plc:123'),
# preserve protocol bot user ids
('https://fed.brid.gy/', 'https://fed.brid.gy/'),
('https://fa.brid.gy/', 'https://fa.brid.gy/'),
('fa.brid.gy', 'fa.brid.gy'),
]:
self.assertEqual(expected, common.unwrap(input))

Wyświetl plik

@ -16,6 +16,7 @@ from .testutil import Fake, TestCase
from activitypub import ActivityPub
from common import unwrap
import ids
from models import Follower, Object
from web import Web
@ -43,17 +44,17 @@ FOLLOWEE = {
FOLLOW_ADDRESS = {
'@context': 'https://www.w3.org/ns/activitystreams',
'type': 'Follow',
'id': f'http://localhost/r/alice.com/following#2022-01-02T03:04:05-@foo@ba.r',
'id': f'http://localhost/r/https://alice.com/#follow-2022-01-02T03:04:05-@foo@ba.r',
'actor': 'http://localhost/alice.com',
'object': FOLLOWEE['id'],
'to': [as2.PUBLIC_AUDIENCE],
}
FOLLOW_URL = copy.deepcopy(FOLLOW_ADDRESS)
FOLLOW_URL['id'] = f'http://localhost/r/alice.com/following#2022-01-02T03:04:05-https://ba.r/actor'
FOLLOW_URL['id'] = f'http://localhost/r/https://alice.com/#follow-2022-01-02T03:04:05-https://ba.r/actor'
UNDO_FOLLOW = {
'@context': 'https://www.w3.org/ns/activitystreams',
'type': 'Undo',
'id': f'http://localhost/r/alice.com/following#undo-2022-01-02T03:04:05-https://ba.r/id',
'id': f'http://localhost/r/https://alice.com/#unfollow-2022-01-02T03:04:05-https://ba.r/id',
'actor': 'http://localhost/alice.com',
'object': copy.deepcopy(FOLLOW_ADDRESS),
}
@ -213,7 +214,7 @@ class FollowTest(TestCase):
follow_with_profile_link = {
**FOLLOW_URL,
'id': f'http://localhost/r/alice.com/following#2022-01-02T03:04:05-https://ba.r/id',
'id': f'http://localhost/r/https://alice.com/#follow-2022-01-02T03:04:05-https://ba.r/id',
'object': 'https://ba.r/id',
}
self.check('https://ba.r/id', resp, follow_with_profile_link, mock_get,
@ -355,7 +356,7 @@ class FollowTest(TestCase):
sig_template.startswith('keyId="http://localhost/alice.com#key"'),
sig_template)
follow_id = f'https://fed.brid.gy/web/alice.com/following#2022-01-02T03:04:05-{input}'
follow_id = f'https://alice.com/#follow-2022-01-02T03:04:05-{input}'
followers = Follower.query().fetch()
followee = ActivityPub(id='https://ba.r/id').key
@ -367,7 +368,10 @@ class FollowTest(TestCase):
if not expected_follow_as1:
expected_follow_as1 = as2.to_as1(unwrap(expected_follow))
expected_follow_as1['actor'] = ids.translate_user_id(
id=expected_follow_as1['actor'], from_=Web, to=Web)
del expected_follow_as1['to']
self.assert_object(follow_id,
users=[self.user.key],
notify=[followee],
@ -410,24 +414,22 @@ class FollowTest(TestCase):
self.assertEqual(302, resp.status_code)
self.assertEqual('/web/www.alice.com/following', resp.headers['Location'])
id = 'www.alice.com/following#2022-01-02T03:04:05-https://ba.r/actor'
id = 'https://www.alice.com/#follow-2022-01-02T03:04:05-https://ba.r/actor'
expected_follow_as1 = as2.to_as1({
**FOLLOW_URL,
'id': id,
'actor': 'https://www.alice.com/',
'actor': 'www.alice.com',
})
del expected_follow_as1['to']
followee = ActivityPub(id='https://ba.r/id').key
follow_obj = self.assert_object(
f'https://fed.brid.gy/web/{id}',
users=[user.key],
notify=[followee],
status='complete',
labels=['user', 'activity'],
source_protocol='ui',
our_as1=expected_follow_as1,
delivered=['http://ba.r/inbox'],
delivered_protocol='activitypub')
follow_obj = self.assert_object(id, users=[user.key],
notify=[followee],
status='complete',
labels=['user', 'activity'],
source_protocol='ui',
our_as1=expected_follow_as1,
delivered=['http://ba.r/inbox'],
delivered_protocol='activitypub')
followers = Follower.query().fetch()
self.assert_entities_equal(
@ -606,14 +608,17 @@ class UnfollowTest(TestCase):
follower = Follower.query().get()
self.assertEqual('inactive', follower.status)
expected_undo_as1 = as2.to_as1(unwrap(expected_undo))
expected_undo_as1['actor'] = ids.translate_user_id(
id=expected_undo_as1['actor'], from_=Web, to=Web)
self.assert_object(
'https://fed.brid.gy/web/alice.com/following#undo-2022-01-02T03:04:05-https://ba.r/id',
'https://alice.com/#unfollow-2022-01-02T03:04:05-https://ba.r/id',
users=[self.user.key],
notify=[ActivityPub(id='https://ba.r/id').key],
status='complete',
source_protocol='ui',
labels=['user', 'activity'],
our_as1=unwrap(as2.to_as1(expected_undo)),
our_as1=expected_undo_as1,
delivered=['http://ba.r/inbox'],
delivered_protocol='activitypub')
@ -646,7 +651,7 @@ class UnfollowTest(TestCase):
self.assertEqual(302, resp.status_code)
self.assertEqual('/web/www.alice.com/following', resp.headers['Location'])
id = 'http://localhost/r/www.alice.com/following#undo-2022-01-02T03:04:05-https://ba.r/id'
id = 'http://localhost/r/https://www.alice.com/#unfollow-2022-01-02T03:04:05-https://ba.r/id'
expected_undo = {
'@context': 'https://www.w3.org/ns/activitystreams',
'type': 'Undo',
@ -669,14 +674,18 @@ class UnfollowTest(TestCase):
follower = Follower.query().get()
self.assertEqual('inactive', follower.status)
expected_undo_as1 = as2.to_as1(unwrap(expected_undo))
expected_undo_as1['actor'] = ids.translate_user_id(
id=expected_undo_as1['actor'], from_=Web, to=Web)
self.assert_object(
'https://fed.brid.gy/web/www.alice.com/following#undo-2022-01-02T03:04:05-https://ba.r/id',
'https://www.alice.com/#unfollow-2022-01-02T03:04:05-https://ba.r/id',
users=[user.key],
notify=[ActivityPub(id='https://ba.r/id').key],
status='complete',
source_protocol='ui',
labels=['user', 'activity'],
our_as1=unwrap(as2.to_as1(expected_undo)),
our_as1=expected_undo_as1,
delivered=['http://ba.r/inbox'],
delivered_protocol='activitypub')

Wyświetl plik

@ -88,14 +88,18 @@ class IdsTest(TestCase):
self.assertEqual(expected, translate_user_id(
id='https://www.user.com/', from_=Web, to=proto))
@patch('ids._FED_SUBDOMAIN_SITES', new={'on-fed.com'})
def test_translate_user_id_web_ap_subdomain_fed(self):
self.make_user('on-fed.com', cls=Web, ap_subdomain='fed')
self.make_user('on-bsky.com', cls=Web, ap_subdomain='bsky')
for base_url in ['https://web.brid.gy/', 'https://fed.brid.gy/']:
with app.test_request_context('/', base_url=base_url):
self.assertEqual('https://web.brid.gy/on-web.com', translate_user_id(
id='on-web.com', from_=Web, to=ActivityPub))
self.assertEqual('https://fed.brid.gy/on-fed.com', translate_user_id(
id='on-fed.com', from_=Web, to=ActivityPub))
self.assertEqual('https://bsky.brid.gy/on-bsky.com', translate_user_id(
id='on-bsky.com', from_=Web, to=ActivityPub))
def test_translate_handle(self):
for from_, handle, to, expected in [
@ -119,6 +123,11 @@ class IdsTest(TestCase):
(Fake, 'fake:handle:user', ATProto, 'fake:handle:user.fa.brid.gy'),
(Fake, 'fake:handle:user', Fake, 'fake:handle:user'),
(Fake, 'fake:handle:user', Web, 'fake:handle:user'),
# instance actor, protocol bot users
(Web, 'fed.brid.gy', ActivityPub, '@fed.brid.gy@fed.brid.gy'),
(Web, 'bsky.brid.gy', ActivityPub, '@bsky.brid.gy@bsky.brid.gy'),
(Web, 'ap.brid.gy', ATProto, 'ap.brid.gy'),
]:
with self.subTest(from_=from_.LABEL, to=to.LABEL):
self.assertEqual(expected, translate_handle(
@ -132,6 +141,10 @@ class IdsTest(TestCase):
(ActivityPub, '@user@user', Web, 'https://user'),
(ActivityPub, '@user@instance', Fake, 'fake:handle:@user@instance'),
(ATProto, 'user.com', ActivityPub, '@user.com@user.com'),
# instance actor, protocol bot user
(Web, 'fed.brid.gy', ActivityPub, '@fed.brid.gy@fed.brid.gy'),
(Web, 'bsky.brid.gy', ActivityPub, '@bsky.brid.gy@bsky.brid.gy'),
]:
with self.subTest(from_=from_.LABEL, to=to.LABEL):
self.assertEqual(expected, translate_handle(
@ -185,8 +198,9 @@ class IdsTest(TestCase):
self.assertEqual(expected, translate_object_id(
id=id, from_=from_, to=to))
@patch('ids._FED_SUBDOMAIN_SITES', new={'on-fed.com'})
def test_translate_object_id_web_ap_subdomain_fed(self):
self.make_user('on-fed.com', cls=Web, ap_subdomain='fed')
for base_url in ['https://web.brid.gy/', 'https://fed.brid.gy/']:
with app.test_request_context('/', base_url=base_url):
got = translate_object_id(id='http://on-fed.com/post', from_=Web,

Wyświetl plik

@ -4,19 +4,21 @@ from unittest.mock import patch
from arroba.datastore_storage import DatastoreStorage
from arroba.repo import Repo
from flask import g
from dns.resolver import NXDOMAIN
from granary import as2
from granary.tests.test_bluesky import ACTOR_PROFILE_BSKY, POST_BSKY
from oauth_dropins.webutil.flask_util import NoContent
from oauth_dropins.webutil.testutil import requests_response
from activitypub import ActivityPub
import app
from atproto import ATProto
from dns.resolver import NXDOMAIN
from granary.tests.test_bluesky import ACTOR_PROFILE_BSKY, POST_BSKY
import hub
from models import Object, Target
from web import Web
from .testutil import ATPROTO_KEY, TestCase
from .test_activitypub import ACTOR
from . import test_atproto
from . import test_web
@ -32,6 +34,7 @@ PROFILE_GETRECORD = {
}
@patch('ids.COPIES_PROTOCOLS', ['atproto'])
class IntegrationTests(TestCase):
@patch('requests.post')
@ -354,3 +357,101 @@ class IntegrationTests(TestCase):
**POST_BSKY,
'cid': 'sydd',
})
@patch('requests.post', return_value=requests_response('OK')) # create DID
@patch('requests.get')
def test_activitypub_follow_bsky_bot_user_enables_protocol(self, mock_get, _):
"""AP follow of @bsky.brid.gy@bsky.brid.gy bridges the account into BLuesky.
ActivityPub user @alice@inst , https://inst/alice
ATProto bot user bsky.brid.gy (did:plc:bsky)
Follow is https://inst/follow
"""
mock_get.return_value = self.as2_resp({
'type': 'Person',
'id': 'https://inst/alice',
'name': 'Mrs. ☕ Alice',
'preferredUsername': 'alice',
'inbox': 'http://inst/inbox',
})
self.make_user(id='bsky.brid.gy', cls=Web, ap_subdomain='bsky')
# deliver follow
resp = self.post('/bsky.brid.gy/inbox', json={
'type': 'Follow',
'id': 'http://inst/follow',
'actor': 'https://inst/alice',
'object': 'https://bsky.brid.gy/bsky.brid.gy',
})
self.assertEqual(204, resp.status_code)
# check results
user = ActivityPub.get_by_id('https://inst/alice')
self.assertTrue(ActivityPub.is_enabled_to(ATProto, user=user))
self.assertEqual(1, len(user.copies))
self.assertEqual('atproto', user.copies[0].protocol)
did = user.copies[0].uri
storage = DatastoreStorage()
repo = storage.load_repo('alice.inst.ap.brid.gy')
self.assertEqual(did, repo.did)
records = repo.get_contents()
self.assertEqual(['app.bsky.actor.profile'], list(records.keys()))
self.assertEqual(['self'], list(records['app.bsky.actor.profile'].keys()))
@patch('requests.post')
@patch('requests.get')
def test_atproto_follow_ap_bot_user_enables_protocol(self, mock_get, mock_post):
"""Bluesky follow of @ap.brid.gy enables the ActivityPub protocol.
ATProto user alice.com, did:plc:alice
ActivityPub bot user @ap.brid.gy, did:plc:ap
"""
self.make_user(id='ap.brid.gy', cls=Web, ap_subdomain='ap',
enabled_protocols=['atproto'],
copies=[Target(uri='did:plc:ap', protocol='atproto')])
self.store_object(id='did:plc:ap', raw={
**DID_DOC,
'id': 'did:plc:ap',
'alsoKnownAs': ['at://ap.brid.gy'],
})
storage = DatastoreStorage()
Repo.create(storage, 'did:plc:ap', signing_key=ATPROTO_KEY)
mock_get.side_effect = [
# ATProto listNotifications
requests_response({
'cursor': '...',
'notifications': [{
'uri': 'at://did:plc:alice/app.bsky.graph.follow/456',
'cid': '...',
'author': {
'$type': 'app.bsky.actor.defs#profileView',
'did': 'did:plc:alice',
'handle': 'alice.com',
},
'reason': 'follow',
'record': {
'$type': 'app.bsky.graph.follow',
'subject': 'did:plc:ap',
},
}],
}),
# alice DID
requests_response(DID_DOC),
# alice profile
requests_response(PROFILE_GETRECORD),
# alice.com handle resolution, HTTPS method
# requests_response('did:plc:alice', content_type='text/plain'),
# # alice profile
# requests_response(PROFILE_GETRECORD),
]
resp = self.post('/queue/atproto-poll-notifs', client=hub.app.test_client())
self.assertEqual(200, resp.status_code)
user = ATProto.get_by_id('did:plc:alice')
self.assertTrue(ATProto.is_enabled_to(ActivityPub, user=user))

Wyświetl plik

@ -66,10 +66,15 @@ class UserTest(TestCase):
user.direct = True
self.assert_entities_equal(same, user, ignore=['updated'])
@patch('ids.COPIES_PROTOCOLS', ['fake', 'other'])
def test_get_or_create_propagate_fake_other(self):
user = Fake.get_or_create('fake:user', propagate=True)
self.assertEqual(['fake:user'], OtherFake.created_for)
@patch.object(tasks_client, 'create_task', return_value=Task(name='my task'))
@patch('requests.post',
return_value=requests_response('OK')) # create DID on PLC
def test_get_or_create_propagate(self, mock_post, mock_create_task):
def test_get_or_create_propagate_atproto(self, mock_post, mock_create_task):
common.RUN_TASKS_INLINE = False
Fake.fetchable = {
@ -107,6 +112,7 @@ class UserTest(TestCase):
mock_create_task.assert_called()
@patch('ids.COPIES_PROTOCOLS', ['eefake', 'atproto'])
@patch.object(tasks_client, 'create_task')
@patch('requests.post')
@patch('requests.get')
@ -123,7 +129,6 @@ class UserTest(TestCase):
self.assertEqual([], user.copies)
self.assertEqual(0, AtpRepo.query().count())
def test_get_or_create_use_instead(self):
user = Fake.get_or_create('a.b')
user.use_instead = self.user.key
@ -133,6 +138,11 @@ class UserTest(TestCase):
self.assertEqual('y.z', got.key.id())
assert got.existing
def test_get_or_create_by_copies(self):
other = self.make_user(id='other:ab', cls=OtherFake,
copies=[Target(uri='fake:ab', protocol='fake')])
self.assert_entities_equal(other, Fake.get_or_create('fake:ab'))
def test_get_or_create_opted_out(self):
user = self.make_user('fake:user', cls=Fake,
obj_as1 = {'summary': '#nobridge'})
@ -278,9 +288,13 @@ class UserTest(TestCase):
user.copies.append(Target(uri='fake:foo', protocol='fake'))
self.assertIsNone(user.get_copy(OtherFake))
self.assertIsNone(user.get_copy(OtherFake))
user.copies = [Target(uri='other:foo', protocol='other')]
self.assertEqual('other:foo', user.get_copy(OtherFake))
self.assertIsNone(OtherFake().get_copy(Fake))
def test_count_followers(self):
self.assertEqual((0, 0), self.user.count_followers())

Wyświetl plik

@ -168,6 +168,24 @@ class PagesTest(TestCase):
got = self.client.get('/web/user.com?before=2024-01-01+01:01:01&after=2023-01-01+01:01:01')
self.assert_equals(400, got.status_code)
def test_update_profile(self):
self.make_user('fake:user', cls=Fake)
actor = {
'objectType': 'person',
'id': 'fake:user',
'displayName': 'Ms User',
}
Fake.fetchable = {'fake:user': actor}
got = self.client.post('/fa/fake:user/update-profile')
self.assert_equals(302, got.status_code)
self.assert_equals('/fa/fake:handle:user', got.headers['Location'])
self.assertEqual(['Updating profile for fake:handle:user'],
get_flashed_messages())
self.assertEqual(['fake:user'], Fake.fetched)
self.assert_object('fake:user', source_protocol='fake', our_as1=actor)
def test_followers(self):
Follower.get_or_create(
to=self.user,

Wyświetl plik

@ -90,8 +90,13 @@ class ProtocolTest(TestCase):
('foo://bar', None),
('fake:foo', Fake),
('at://foo', ATProto),
# TODO: remove? should we require normalized ids?
('https://ap.brid.gy/foo/bar', ActivityPub),
('https://web.brid.gy/foo/bar', Web),
('https://fed.brid.gy/', Web),
('https://web.brid.gy/', Web),
('https://bsky.brid.gy/', Web),
('bsky.brid.gy', Web),
]:
self.assertEqual(expected, Protocol.for_id(id, remote=False))
self.assertEqual(expected, Protocol.for_id(id, remote=True))
@ -109,7 +114,8 @@ class ProtocolTest(TestCase):
self.store_object(id='http://u.i/obj', source_protocol='ui')
self.assertEqual(UIProtocol, Protocol.for_id('http://u.i/obj'))
def test_for_id_object_missing_source_protocol(self):
@patch('requests.get', return_value=requests_response())
def test_for_id_object_missing_source_protocol(self, _):
self.store_object(id='http://ba.d/obj')
self.assertIsNone(Protocol.for_id('http://ba.d/obj'))
@ -789,7 +795,6 @@ class ProtocolReceiveTest(TestCase):
def test_update_post_bare_object(self):
self.make_followers()
# post has no author
post_as1 = {
'id': 'fake:post',
'objectType': 'note',
@ -1293,6 +1298,43 @@ class ProtocolReceiveTest(TestCase):
self.assertEqual([(update_obj.key.id(), 'shared:target')], Fake.sent)
def test_update_profile_bare_object(self):
self.make_followers()
actor = {
'objectType': 'person',
'id': 'fake:user',
'displayName': 'Ms. ☕ Baz',
'summary': 'first',
}
self.store_object(id='fake:user', our_as1=actor)
actor['summary'] = 'second'
Fake.receive_as1(actor)
# profile object
actor['updated'] = '2022-01-02T03:04:05+00:00'
self.assert_object('fake:user', our_as1=actor, type='person')
# update activity
id = 'fake:user#bridgy-fed-update-2022-01-02T03:04:05+00:00'
update_obj = self.assert_object(
id,
users=[self.user.key],
status='complete',
our_as1={
'objectType': 'activity',
'verb': 'update',
'id': id,
'actor': actor,
'object': actor,
},
delivered=['shared:target'],
type='update',
object_ids=['fake:user'],
)
self.assertEqual([(id, 'shared:target')], Fake.sent)
def test_mention_object(self, *mocks):
self.alice.obj.our_as1 = {'id': 'fake:alice', 'objectType': 'person'}
self.alice.obj.put()
@ -1607,6 +1649,27 @@ class ProtocolReceiveTest(TestCase):
self.assertEqual(1, len(followers))
self.assertEqual(self.alice.key, followers[0].to)
def test_skip_bridged_user(self):
"""If the actor isn't from the source protocol, skip the activity.
(It's probably from a bridged user, and we only want to handle source
activities, not bridged activities.)
"""
self.user.copies = [Target(uri='other:user', protocol='other')]
self.user.put()
with self.assertRaises(NoContent):
OtherFake.receive_as1({
'id': 'other:follow',
'objectType': 'activity',
'verb': 'follow',
'actor': 'fake:user',
'object': 'fake:alice',
})
self.assertEqual(0, len(OtherFake.sent))
self.assertEqual(0, len(Fake.sent))
self.assertIsNone(Object.get_by_id('other:follow'))
@patch('requests.post')
@patch('requests.get')
def test_skip_web_same_domain(self, mock_get, mock_post):
@ -1800,30 +1863,34 @@ class ProtocolReceiveTest(TestCase):
self.assertEqual(('OK', 200), ExplicitEnableFake.receive_as1(block))
user = user.key.get()
self.assertEqual([], user.enabled_protocols)
self.assertEqual([], Fake.created_for)
# follow should add to enabled_protocols
with self.assertRaises(NoContent):
ExplicitEnableFake.receive_as1(follow)
user = user.key.get()
self.assertEqual(['fake'], user.enabled_protocols)
self.assertEqual(['eefake:user'], Fake.created_for)
self.assertTrue(ExplicitEnableFake.is_enabled_to(Fake, user))
self.assertEqual([
('https://fa.brid.gy//followers#accept-eefake:follow',
'eefake:user:target'),
], ExplicitEnableFake.sent)
self.assertEqual([('fa.brid.gy/followers#accept-eefake:follow',
'eefake:user:target')],
ExplicitEnableFake.sent)
# another follow should be a noop
follow['id'] += '2'
Fake.created_for = []
with self.assertRaises(NoContent):
ExplicitEnableFake.receive_as1(follow)
user = user.key.get()
self.assertEqual(['fake'], user.enabled_protocols)
self.assertEqual([], Fake.created_for)
# block should remove from enabled_protocols
block['id'] += '2'
self.assertEqual(('OK', 200), ExplicitEnableFake.receive_as1(block))
user = user.key.get()
self.assertEqual([], user.enabled_protocols)
self.assertEqual([], Fake.created_for)
self.assertFalse(ExplicitEnableFake.is_enabled_to(Fake, user))
def test_dm_no_yes_sets_enabled_protocols(self):
@ -1842,27 +1909,32 @@ class ProtocolReceiveTest(TestCase):
self.assertEqual(('OK', 200), ExplicitEnableFake.receive_as1(dm))
user = user.key.get()
self.assertEqual([], user.enabled_protocols)
self.assertEqual([], Fake.created_for)
# yes DM should add to enabled_protocols
dm['id'] += '2'
dm['content'] = 'yes'
dm['content'] = '<p><a href="...">@bsky.brid.gy</a> yes</p>'
self.assertEqual(('OK', 200), ExplicitEnableFake.receive_as1(dm))
user = user.key.get()
self.assertEqual(['fake'], user.enabled_protocols)
self.assertEqual(['eefake:user'], Fake.created_for)
self.assertTrue(ExplicitEnableFake.is_enabled_to(Fake, user))
# another yes DM should be a noop
dm['id'] += '3'
Fake.created_for = []
self.assertEqual(('OK', 200), ExplicitEnableFake.receive_as1(dm))
user = user.key.get()
self.assertEqual(['fake'], user.enabled_protocols)
self.assertEqual([], Fake.created_for)
# block should remove from enabled_protocols
# no DM should remove from enabled_protocols
dm['id'] += '4'
dm['content'] = ' \n NO '
dm['content'] = '<p><a href="...">@bsky.brid.gy</a>\n NO \n</p>'
self.assertEqual(('OK', 200), ExplicitEnableFake.receive_as1(dm))
user = user.key.get()
self.assertEqual([], user.enabled_protocols)
self.assertEqual([], Fake.created_for)
self.assertFalse(ExplicitEnableFake.is_enabled_to(Fake, user))
def test_receive_task_handler(self):

Wyświetl plik

@ -208,7 +208,7 @@ REPLY = requests_response(REPLY_HTML, url='https://user.com/reply')
REPLY_MF2 = util.parse_mf2(REPLY_HTML)['items'][0]
REPLY_AS1 = microformats2.json_to_object(REPLY_MF2)
REPLY_AS1['id'] = 'https://user.com/reply'
REPLY_AS1['author']['id'] = 'https://user.com/'
REPLY_AS1['author']['id'] = 'user.com'
CREATE_REPLY_AS1 = {
'objectType': 'activity',
'verb': 'post',
@ -351,13 +351,16 @@ NOTE_HTML = """\
NOTE = requests_response(NOTE_HTML, url='https://user.com/post')
NOTE_MF2 = util.parse_mf2(NOTE_HTML)['items'][0]
NOTE_AS1 = microformats2.json_to_object(NOTE_MF2)
NOTE_AS1.update({
'author': {
**NOTE_AS1['author'],
'id': 'https://user.com/',
},
'id': 'https://user.com/post',
})
NOTE_AS1['id'] = 'https://user.com/post'
NOTE_AS1['author']['id'] = 'user.com'
CREATE_AS1 = {
'objectType': 'activity',
'verb': 'post',
'id': 'https://user.com/post#bridgy-fed-create',
'actor': ACTOR_AS1_UNWRAPPED,
'object': copy.deepcopy(NOTE_AS1),
'published': '2022-01-02T03:04:05+00:00',
}
NOTE_AS2 = {
'type': 'Note',
'id': 'http://localhost/r/https://user.com/post',
@ -368,14 +371,6 @@ NOTE_AS2 = {
'contentMap': {'en': 'hello i am a post'},
'to': [as2.PUBLIC_AUDIENCE],
}
CREATE_AS1 = {
'objectType': 'activity',
'verb': 'post',
'id': 'https://user.com/post#bridgy-fed-create',
'actor': ACTOR_AS1_UNWRAPPED,
'object': NOTE_AS1,
'published': '2022-01-02T03:04:05+00:00',
}
CREATE_AS2 = {
'@context': 'https://www.w3.org/ns/activitystreams',
'type': 'Create',
@ -481,11 +476,11 @@ class WebTest(TestCase):
'acct:user.com',
'acct:@user.com@user.com',
'acc:me@user.com',
'ap.brid.gy',
'localhost',
):
with self.assertRaises(AssertionError):
Web(id=bad).put()
with self.subTest(id=bad):
with self.assertRaises(AssertionError):
Web(id=bad).put()
def test_get_or_create_lower_cases_domain(self, mock_get, mock_post):
mock_get.return_value = requests_response('')
@ -1150,10 +1145,12 @@ class WebTest(TestCase):
self.assertEqual(202, got.status_code)
inboxes = ['https://inbox/', 'https://public/inbox', 'https://shared/inbox']
expected_create_as1 = copy.deepcopy(CREATE_AS1)
expected_create_as1['object']['author']['id'] = 'https://user.com/'
self.assert_object('https://user.com/post#bridgy-fed-create',
users=[self.user.key],
source_protocol='web',
our_as1=CREATE_AS1,
our_as1=expected_create_as1,
type='post',
labels=['activity', 'user'],
delivered=inboxes,
@ -1632,6 +1629,10 @@ class WebTest(TestCase):
Follower.get_or_create(to=self.user, from_=self.make_user(
'http://d/dd', cls=ActivityPub, obj_as2={'inbox': 'https://inbox'}))
mf2 = copy.deepcopy(ACTOR_MF2)
mf2['properties']['name'] = 'original'
self.store_object(id='https://user.com/', mf2=mf2)
got = self.post('/queue/webmention', data={
'source': 'https://user.com/',
'target': 'https://fed.brid.gy/',
@ -1641,7 +1642,7 @@ class WebTest(TestCase):
self.req('https://user.com/'),
))
id = 'https://user.com/#update-2022-01-02T03:04:05+00:00'
id = 'https://user.com/#bridgy-fed-update-2022-01-02T03:04:05+00:00'
wrapped_id = f'http://localhost/r/{id}'
expected_as2 = {
'@context': 'https://www.w3.org/ns/activitystreams',
@ -1730,7 +1731,7 @@ class WebTest(TestCase):
self.req('https://user.com/'),
))
id = 'https://user.com/#update-2022-01-02T03:04:05+00:00'
id = 'https://user.com/#bridgy-fed-update-2022-01-02T03:04:05+00:00'
wrapped_id = f'http://localhost/r/{id}'
update_as2 = {
'@context': 'https://www.w3.org/ns/activitystreams',
@ -2373,6 +2374,13 @@ http://this/404s
self.user.ap_subdomain = 'fed'
self.assertEqual('@user.com@fed.brid.gy', self.user.handle_as(ActivityPub))
def test_handle_as_bot_users(self, *_):
fed = Web(id='fed.brid.gy', ap_subdomain='fed')
self.assertEqual('@fed.brid.gy@fed.brid.gy', fed.handle_as(ActivityPub))
bsky = Web(id='bsky.brid.gy', ap_subdomain='bsky')
self.assertEqual('@bsky.brid.gy@bsky.brid.gy', bsky.handle_as(ActivityPub))
def test_id_as(self, *_):
self.assertEqual('http://localhost/user.com', self.user.id_as(ActivityPub))
@ -2552,7 +2560,9 @@ class WebUtilTest(TestCase):
self.assertEqual(False, Web.owns_handle('@foo@bar.com'))
self.assertEqual(False, Web.owns_handle('foo@bar.com'))
self.assertEqual(False, Web.owns_handle('localhost'))
self.assertEqual(False, Web.owns_handle('bsky.brid.gy'))
self.assertEqual(True, Web.owns_handle('fed.brid.gy'))
self.assertEqual(True, Web.owns_handle('bsky.brid.gy'))
def test_handle_to_id(self, *_):
self.assertEqual('foo.com', Web.handle_to_id('foo.com'))
@ -2755,8 +2765,7 @@ class WebUtilTest(TestCase):
def test_fetch_instance_actor(self, _, __):
obj = Object(id=f'https://{common.PRIMARY_DOMAIN}/')
self.assertTrue(Web.fetch(obj))
self.assertEqual(obj.as2,
json_loads(util.read('static/instance-actor.as2.json')))
self.assertEqual(obj.as2, json_loads(util.read('fed.brid.gy.as2.json')))
def test_fetch_resolves_relative_urls(self, mock_get, __):
mock_get.return_value = requests_response("""\

Wyświetl plik

@ -4,10 +4,11 @@ from unittest.mock import patch
import urllib.parse
from granary.as2 import CONTENT_TYPE_LD_PROFILE
from oauth_dropins.webutil import util
from oauth_dropins.webutil.testutil import requests_response
# import first so that Fake is defined before URL routes are registered
from .testutil import Fake, TestCase
from .testutil import ExplicitEnableFake, Fake, TestCase
from models import PROTOCOLS
import protocol
@ -295,6 +296,17 @@ class WebfingerTest(TestCase):
got = self.client.get(f'/.well-known/webfinger?resource=acct:user.com@user.com')
self.assertEqual(404, got.status_code)
def test_protocol_not_enabled(self):
self.make_user('eefake:user', cls=ExplicitEnableFake)
got = self.client.get(f'/.well-known/webfinger?resource=acct:eefake:user@eefake.brid.gy')
self.assertEqual(404, got.status_code)
def test_protocol_enabled(self):
self.make_user('eefake:user', cls=ExplicitEnableFake,
enabled_protocols=['activitypub'])
got = self.client.get(f'/.well-known/webfinger?resource=acct:eefake:user@eefake.brid.gy')
self.assertEqual(200, got.status_code)
def test_bad_id(self):
got = self.client.get(f'/.well-known/webfinger?resource=acct:nope@fa.brid.gy')
self.assertEqual(400, got.status_code, got.get_data(as_text=True))
@ -341,11 +353,29 @@ class WebfingerTest(TestCase):
user = Web.get_by_id('user.com')
assert not user.direct
def test_fed_brid_gy(self):
# skip _pre_put_hook since it doesn't allow internal domains
@patch.object(Web, '_pre_put_hook', new=lambda self: None)
def test_protocol_bot_user(self):
self.make_user('bsky.brid.gy', cls=Web, obj_id='https://bsky.brid.gy/',
ap_subdomain='bsky')
for id in ('acct:bsky.brid.gy@bsky.brid.gy',
'https://bsky.brid.gy/bsky.brid.gy'):
got = self.client.get(f'/.well-known/webfinger?resource={id}')
self.assertEqual(200, got.status_code, got.get_data(as_text=True))
self.assertEqual('acct:bsky.brid.gy@bsky.brid.gy', got.json['subject'])
self.assertEqual(['https://bsky.brid.gy/'], got.json['aliases'])
self.assertIn({
'href': 'http://localhost/bsky.brid.gy',
'rel': 'self',
'type': 'application/ld+json; profile="https://www.w3.org/ns/activitystreams"',
}, got.json['links'])
def test_internal_domain_error(self):
got = self.client.get('/.well-known/webfinger?resource=http://localhost/')
self.assertEqual(400, got.status_code, got.get_data(as_text=True))
got = self.client.get('/.well-known/webfinger?resource=acct%3A%40localhost')
got = self.client.get('/.well-known/webfinger?resource=acct:@localhost')
self.assertEqual(400, got.status_code, got.get_data(as_text=True))
@patch('requests.get', return_value=requests_response(

Wyświetl plik

@ -34,7 +34,8 @@ import requests
# other modules are imported _after_ Fake etc classes is defined so that it's in
# PROTOCOLS when URL routes are registered.
from common import long_to_base64, TASKS_LOCATION
from common import add, long_to_base64, TASKS_LOCATION
import ids
import models
from models import KEY_BITS, Object, PROTOCOLS, Target, User
import protocol
@ -69,6 +70,7 @@ class Fake(User, protocol.Protocol):
ABBREV = 'fa'
PHRASE = 'fake-phrase'
CONTENT_TYPE = 'fa/ke'
HAS_COPIES = True
# maps string ids to dict AS1 objects that can be fetched
fetchable = {}
@ -76,8 +78,9 @@ class Fake(User, protocol.Protocol):
# in-order list of (Object, str URL)
sent = []
# in-order list of ids
# in-order lists of ids
fetched = []
created_for = []
@ndb.ComputedProperty
def handle(self):
@ -86,6 +89,14 @@ class Fake(User, protocol.Protocol):
def web_url(self):
return self.key.id()
@classmethod
def create_for(cls, user):
assert not user.get_copy(cls)
id = user.key.id()
cls.created_for.append(id)
add(user.copies, Target(uri=ids.translate_user_id(id=id, from_=user, to=cls),
protocol=cls.LABEL))
@classmethod
def owns_id(cls, id):
if id.startswith('nope') or id == f'{cls.LABEL}:nope':
@ -106,7 +117,7 @@ class Fake(User, protocol.Protocol):
return handle.replace(f'{cls.LABEL}:handle:', f'{cls.LABEL}:')
@classmethod
def is_blocklisted(cls, url):
def is_blocklisted(cls, url, allow_internal=False):
return url.startswith(f'{cls.LABEL}:blocklisted')
@classmethod
@ -157,6 +168,7 @@ class OtherFake(Fake):
fetchable = {}
sent = []
fetched = []
created_for = []
@classmethod
def target_for(cls, obj, shared=False):
@ -171,6 +183,7 @@ class ExplicitEnableFake(Fake):
fetchable = {}
sent = []
fetched = []
created_for = []
# import other modules that register Flask handlers *after* Fake is defined
@ -181,6 +194,7 @@ import activitypub
from activitypub import ActivityPub, CONNEG_HEADERS_AS2_HTML
from atproto import ATProto
import common
from common import PRIMARY_DOMAIN, PROTOCOL_DOMAINS, OTHER_DOMAINS, LOCAL_DOMAINS
from web import Web
from flask_app import app, cache
@ -211,13 +225,13 @@ class TestCase(unittest.TestCase, testutil.Asserts):
did.resolve_plc.cache.clear()
did.resolve_web.cache.clear()
for cls in Fake, OtherFake:
for cls in ExplicitEnableFake, Fake, OtherFake:
cls.fetchable = {}
cls.sent = []
cls.fetched = []
cls.created_for = []
common.OTHER_DOMAINS += ('fake.brid.gy',)
common.DOMAINS += ('fake.brid.gy',)
ids._NON_WEB_SUBDOMAIN_SITES = None
# make random test data deterministic
arroba.util._clockid = 17

93
web.py
Wyświetl plik

@ -24,7 +24,7 @@ from requests.auth import HTTPBasicAuth
from werkzeug.exceptions import BadGateway, BadRequest, HTTPException, NotFound
import common
from common import add, DOMAIN_RE, SUPERDOMAIN
from common import add, DOMAIN_RE, PRIMARY_DOMAIN, PROTOCOL_DOMAINS, SUPERDOMAIN
from flask_app import app, cache
from ids import translate_handle, translate_object_id, translate_user_id
from models import Follower, Object, PROTOCOLS, Target, User
@ -67,16 +67,21 @@ MAX_FEED_POLL_PERIOD = timedelta(weeks=1)
MAX_FEED_PROPERTY_SIZE = 500 * 1000 # Object.atom/rss
def is_valid_domain(domain):
def is_valid_domain(domain, allow_internal=True):
"""Returns True if this is a valid domain we can use, False otherwise.
Args:
domain (str):
allow_internal (bool): whether to return True for internal domains
like ``fed.brid.gy``, ``bsky.brid.gy``, etc
Valid means TLD is ok, not blacklisted, etc.
"""
if not domain or not re.match(DOMAIN_RE, domain):
# logger.debug(f"{domain} doesn't look like a domain")
return False
if Web.is_blocklisted(domain) and domain != common.PRIMARY_DOMAIN:
if Web.is_blocklisted(domain, allow_internal=allow_internal):
logger.debug(f'{domain} is blocklisted')
return False
@ -112,9 +117,14 @@ class Web(User, Protocol):
# Originally, BF served Web users' AP actor ids on fed.brid.gy, eg
# https://fed.brid.gy/snarfed.org . When we started adding new protocols, we
# switched to per-protocol subdomains, eg https://web.brid.gy/snarfed.org .
# However, we need to preserve the old users' actor ids as is. So, this
# property tracks which subdomain a given Web user's AP actor uses.
ap_subdomain = ndb.StringProperty(choices=['fed', 'web'], default='web')
# However, we need to preserve the old users' actor ids as is.
#
# Also, our per-protocol bot accounts in ActivityPub are on their own
# subdomains, eg @bsky.brid.gy@bsky.brid.gy.
#
# So, this property tracks which subdomain a given Web user's AP actor uses.
ap_subdomain = ndb.StringProperty(choices=['ap', 'bsky', 'fed', 'web'],
default='web')
# OLD. some stored entities still have these; do not reuse.
# superfeedr_subscribed = ndb.DateTimeProperty(tzinfo=timezone.utc)
@ -317,7 +327,7 @@ class Web(User, Protocol):
if parsed.path in ('', '/'):
id = parsed.netloc
if is_valid_domain(id):
if is_valid_domain(id, allow_internal=True):
return super().key_for(id)
# logger.info(f'{id} is not a domain or usable home page URL')
@ -337,14 +347,21 @@ class Web(User, Protocol):
return True if user and user.has_redirects else None
elif is_valid_domain(id):
return None
elif util.is_web(id) and is_valid_domain(util.domain_from_link(id)):
# we allowed internal domains for protocol bot actors above, but we
# don't want to allow non-homepage URLs on those domains, eg
# https://bsky.brid.gy/foo, so don't allow internal here
domain = util.domain_from_link(id)
if util.is_web(id) and is_valid_domain(domain, allow_internal=False):
return None
return False
@classmethod
def owns_handle(cls, handle):
if not is_valid_domain(handle):
if handle == PRIMARY_DOMAIN or handle in PROTOCOL_DOMAINS:
return True
elif not is_valid_domain(handle, allow_internal=False):
return False
@classmethod
@ -440,10 +457,11 @@ class Web(User, Protocol):
return False
is_homepage = urlparse(url).path.strip('/') == ''
if is_homepage and util.domain_from_link(url) == common.PRIMARY_DOMAIN:
obj.as2 = json_loads(util.read('static/instance-actor.as2.json'))
return True
if is_homepage:
domain = util.domain_from_link(url)
if domain == PRIMARY_DOMAIN or domain in PROTOCOL_DOMAINS:
obj.as2 = json_loads(util.read(f'{domain}.as2.json'))
return True
require_backlink = (common.host_url().rstrip('/')
if check_backlink and not is_homepage
@ -578,7 +596,7 @@ def check_web_site():
# this normalizes and lower cases domain
domain = util.domain_from_link(url, minimize=False)
if not domain or not is_valid_domain(domain):
if not domain or not is_valid_domain(domain, allow_internal=False):
flash(f'{url} is not a valid or supported web site')
return render_template('enter_web_site.html'), 400
@ -626,32 +644,12 @@ def webmention_external():
if not user:
error(f'No user found for domain {domain}')
if request.path == '/webmention': # exclude interactive
user.last_webmention_in = util.now()
user.put()
user.last_webmention_in = util.now()
user.put()
return common.create_task('webmention', **request.form)
@app.post('/webmention-interactive')
def webmention_interactive():
"""Handler that runs interactive webmention-based requests from the web UI.
...eg the update profile button on user pages.
"""
source = flask_util.get_required_param('source').strip()
try:
webmention_external()
user = Web(id=util.domain_from_link(source, minimize=False))
flash(f'Updating profile from <a href="{user.web_url()}">{user.key.id()}</a>...')
return redirect(user.user_page_path(), code=302)
except HTTPException as e:
flash(util.linkify(str(e.description), pretty=True))
return redirect('/', code=302)
@app.post(f'/queue/poll-feed')
@cloud_tasks_only
def poll_feed_task():
@ -841,29 +839,6 @@ def webmention_task():
else:
authors[0] = user.web_url()
# if source is home page, update Web user and send an actor Update to
# followers' instances
if user.key.id() == obj.key.id() or user.is_web_url(obj.key.id()):
logger.info(f'Converted to AS1: {obj.type}: {json_dumps(obj.as1, indent=2)}')
obj.put()
user.obj = obj
user.put()
logger.info('Wrapping in Update for home page user profile')
actor_as1 = {
**obj.as1,
'id': user.web_url(),
'updated': util.now().isoformat(),
}
id = common.host_url(f'{obj.key.id()}#update-{util.now().isoformat()}')
obj = Object(id=id, status='new', our_as1={
'objectType': 'activity',
'verb': 'update',
'id': id,
'actor': user.web_url(),
'object': actor_as1,
})
try:
return Web.receive(obj, authed_as=user.web_url())
except ValueError as e:

Wyświetl plik

@ -14,9 +14,10 @@ from oauth_dropins.webutil.util import json_dumps, json_loads
import activitypub
import common
from common import LOCAL_DOMAINS, SUPERDOMAIN
from common import LOCAL_DOMAINS, PRIMARY_DOMAIN, PROTOCOL_DOMAINS, SUPERDOMAIN
from flask_app import app, cache
from protocol import Protocol
from web import Web
SUBSCRIBE_LINK_REL = 'http://ostatus.org/schema/1.0/subscribe'
@ -58,7 +59,9 @@ class Webfinger(flask_util.XrdOrJrd):
except ValueError:
id = urlparse(resource).netloc or resource
if not cls:
if id == PRIMARY_DOMAIN or id in PROTOCOL_DOMAINS:
cls = Web
elif not cls:
cls = Protocol.for_request(fed='web')
if not cls:
@ -87,7 +90,7 @@ class Webfinger(flask_util.XrdOrJrd):
if user and not user.direct:
error(f"{user.key} hasn't signed up yet", status=404)
if not user:
if not user or not user.is_enabled_to(activitypub.ActivityPub, user=user):
error(f'No {cls.LABEL} user found for {id}', status=404)
ap_handle = user.handle_as('activitypub')