ADC Extensions

Fredrik Ullner <ullner@gmail.com>, 2014-02-11

Fredrik Ullner <ullner@gmail.com>, 2012-11-22

Fredrik Ullner <ullner@gmail.com>, 2010-09-29

Fredrik Ullner <ullner@gmail.com>, 2010-09-16

Fredrik Ullner <ullner@gmail.com>, 2010-06-29

Fredrik Ullner <ullner@gmail.com>, 2010-05-26

Fredrik Ullner <ullner@gmail.com>, 2010-04-04

Fredrik Ullner <ullner@gmail.com>, 2009-08-04

Jacek Sieka <arnetheduck@gmail.com>, 2008-05-02

This extension adds a special file "files.xml.bz2" in the unnamed root of the share which contains "files.xml" compressed with bzip2 1.0.3+ (www.bzip.org).

There are two variants of zlib support, FULL and GET, and only one should be used on a each communications channel set up.

This extension can be supported by both clients and hubs, and when present, if hub supports it, it must send additional information to the client ( otherwise normal base client).

It’s purpose is to send to hublist pingers additional information about the hub that otherwise it would be impossible to get as a normal user (eg. minimum share, maximum user count, etc).

Timestamp of the moment when the message was sent, expressed in seconds since the Unix Epoch (January 1 1970 00:00:00 GMT).

The idea behind this extension is to generate a public hublist from the users favorite hublist. Implementations should separate between public and private hubs in the favorite hublist of an user, in order not to distribute private hubs where one can not connect to anyway.

Signal DFAV in SUP and the INF’s SU field.

Contexts: F

States: NORMAL

User commands are used to send hub-specific commands to the client which provide useful shortcuts for the user. These commands contain strings which must be sent back to the hub and keyword substitutions in the strings. Each user command has a display name, a string to be sent to the hub, and one or more categories where it may appear. The strings passed to the hub must first be passed through a dictionary replacement that replaces all keywords in the string and then through the equivalent of the C standard function "strftime", with the current time.

Name uniquely (per hub) identifies a particular user command. The name may contain "/" to indicate a logical structure on the viewing client, where each "/" introduces a new submenu level. Other than name, the command also has a number of flags that further detail what to do with it.

Bloom filters allow the hub to filter certain searches using bitmap that represents the hashes of the files in the users share. BLOM is an extension that allows hub software to create a map (bloom filter) of the shared files on the hub, but with minimal effort, e.g. the hub doesn’t keep a list of files, but a filter that never produces false negatives but only possible false positives. This can potentially save bandwidth and effort on the client side. When the user updates the share, the client must send an INF containing the flag SF. The hub may at any time request that the client sends an updated version of its bloom filter by sending a GET command to the client. The client will then respond using SND and send the bloom filter binary data.

NAT traversal allow two passive clients to connect to each other. This specification is based on the TCP hole punching algorithm described in
[B. Ford and P. Srisuresh and and D. Kegel. "Peer-to-Peer Communication Across Network Address Translators". In USENIX Technical Conference 2005 - pages 179–192. Online version: http://www.brynosaurus.com/pub/net/p2pnat/]
.

If a client does not support TCP4 or TCP6, it will send an RCM to the client it is trying to connect to. If the other client also doesn’t support TCP4 (or TCP6 correspondingly), NAT traversal may instead be used. Signal NATT in the INF’s SU field.

Do note that the hub must forward I4 or I6 for respective clients' INF.

An endpoint is the tuple of IP and port. The "private endpoint port" refers to the outbound port to the connected hub, as seen by the client. Each client must listen for incoming connections on this port. Note that this protocol extension uses only this port for the TCP hole punching, the use of the "public endpoint port" as specified in
[Peer-to-Peer Communication Across Network Address Translators]
is not supported.

Extends the RF field of the INF to STA, allowing a client to notify clients and hubs upon SUP-negotiation from where the C-C or C-H originated from.

This extension’s purpose is creating a queue on a client, when multiple other clients want to download from it, but they have no slots. Currently, when a slot is being freed, the first connecting client gets it. Other clients that don’t have the luck of getting in time to attempt to download, have to wait again. The client who creates a queue must have a ticket number for each connecting client, which must be kept internally , and a difference between current connecting client’s queue number and the currently uploading client’s be provided to the connecting client, so that the clients are being deserved in the order they originally connected. The client could have a ticket incrementing starting from 1 for each session. Connecting clients must use the same token as they used when originally connected.

Partial File Sharing allows sharing of files which are available in user’s download queue or in finished downloads list. As a result of this, new files will be spread much faster over whole network.

When client receives search request (SCH), it looks into shared list to see whether requested hash is available between shared files. If it’s found, everything is processed as normally. In other case, it looks into download queue (then, into finished downloads list) and receives a list of available chunks for requested hash. It mustn’t take every chunk, but only that ones which are fully downloaded and has already been verified (e.g. against TTH leaves).

The feature should be signalled in SUP as PFSR.

This extension’s purpose is to notify the hub which user locale the client is using as well as the default locale for the hub. This allows hubs to customize text sent to clients, depending on language, left-to-right or right-to-left and more. If the hub does not directly support the client’s locale, it should attempt to fall back to the same language group (e.g. hub supports en-US but not en-AU, so falls back to en-US), and if this is not available, then fall back to the hub’s own locale.

BCP47 should be used as reference for locale structure.

Note that the standard suggest that the language should be in lowercase and the country in upper case. Note that the country code may be more than two characters. Additionally, dash (-) and underscore (_) are acceptable seperators.

This extension will add to the CT field enumeration in the INF to denote a user as "hidden". Other clients shall as appropriate not display the user in user lists etc.

Full updated and comparable text;

This extension will add "Invalid feature" as error code in STA. Invalid features are features the hub or client deem inappropriate or simply not welcome. The error code should not be used for features the hub or client does not know of.

This extension adds a simple, but secure way to protect against man-in-the-middle attacks against ADC when wrapped with TLS (1.0 or later). It does not require setting up a CA or signing keys, but that is still possible if desired.

The extension introduces a keyprint parameter to the ADCS URI. The keyprint parameter is a hash of the server certificate.

The extension also requires clients to publish their own certificates' keyprint in the KP field in the INF. Assuming one trusts the hub enough not to maliciously change the keyprints en route (a reasonable assumption given the hub’s existing position of trust), and given that the connection to the hub has been similarly authenticated (either as above or via a directly downloaded trusted certificate), client-client connections are also protected against attempted man-in-the-middle attacks - without messing around with having to get everyone’s certificates signed in advance.

The keyprint parameter consists of a hash name, followed by a forward slash (/), followed by the Base32-encoded cyrptographic hash of the certificate.

The hash used shall be SHA256. Other extensions may add other hashes, given sufficient security contemplation.

INF field;

This is an extension that allows UDP traffic to be encrypted.

While assymetric encryption may be optimal in sense of security, a symmetric cipher will protect perfectly against outside adversaries given the hub-client connections is also running ADCS. New is that senders now create a random IV for their "request command" (e.g. searches) and send it along the "response command" (e.g. search result).

Signal SUDP in SUP and in the INF’s SU field.

If a client signal support for SUDP in an ADCS hub, it may extend commands that will generate a response (e.g. SCH) with a KY-field as the encryption key. Clients shall only include the flag in ADCS hubs.

For example, a SCH command will result in 29 Bytes of overhead ("<separator>KY"+26 Bytes Base32 encoded key).

A client that has a response for the command can now encrypt the response message by prepending 16 bytes of random data and afterwards encrypting it with AES/CBC/PKCS5Padding (Cipher/Blockmode/Padding) using 16 zero bytes as IV for CBC.

In above scenario, the response would be a RES command.

This extension adds a typing similar to Jabber’s "Chat state notifications".

Signal TYPE in SUP and the INF’s SU field.

The extension adds RSS feed support.

Signal FEED in SUP and the INF’s SU field.

In BASE, clients add EX fields to SCH to denote which extension files should have. This can lead to a situation where the large bulk of extensions are of similar "type", e.g. audio files or documents. This extension intend to add a field GR which groups multiple extensions. In addition, the field RX shall be used for group-exclusion; if all extensions in a group but two are desired, field RX will be used to exclude those group items.

Signal SEGA in the INF’s SU field for support of this extension.

Field GR values, where multiple groups are specified by adding the numbers together:

Field RX:

If a hub goes down, the client’s only option is to keep re-trying the last known hub address. This extension will add a list of failover hub addresses, field FO, that the client can try to connect to, if the main hub address fail.

Clients should decide the frequency of connection attempts (for the main hub as well as the failover addresses). The client should try connecting in the specified order. The client may decide what to do after the FO-list is exhausted, but recommended is to try to connect to the main hub and continue with the list as before.

The client should display an appropriate message to the user that it has connected to a failover address.

This extension should be implemented, at a minimum, for favourite hubs. If clients have any concerns about where a hub may redirect users to, or about storing the extra state information, then it may avoid implementing this for non-favourite hubs.

Field FO in the hub’s INF:

This is an extension that will broadcast the amount of free slots the client has available.

Field FS in the client’s INF:

ADCS is an extension that has the goal of adding the TLS/SSL layer just over the TCP layer and beneath the application layer (where ADC runs). This way, the ADC protocol remains unchanged while the connections are encrypted. The connecting party performs a TLS handshake immediately after the TCP connection is established. The ADC handshake is performed and once the TLS connection is established the ADC handshake proceeds as usual.

Encrypted ADC connections can be established using a TLS tunnel, both for hub and for client connections. Certificates can be used to authenticate both hub and user, for example by making the hub the root CA, and only allow clients signed by the hub to connect. Ephemeral keys should be use to ensure forward secrecy when possible. A future extension or revision of this extension will provide ways to handle certificate based logins, who creates which certificates and who signs what, and all that is not specified in this revision.

This extension adds the parameter AP to the INF to signal application. The current parameter in BASE, VE, will be used for version signalling.

Example:

The purpose of the OID and OIR commands is to allow users to exchange information about their accounts on various online services. The OID name has been intentionally chosen to be similar to OpenID, although its spread is broader.

New commands are preferrable here over new INF fields to avoid clogging up its 2-letter identifiers. They also allow for more flexibility: multiple profiles of the same type (OIR only); multiple identification parameters.

The OID command is similar to INF, guaranteeing up-to-date information, whereas OIR follows a request/response scheme. OIR parties are therefore not required to keep their OIR information up-to-date.

Parties supporting OID commands MUST advertise the ONID (ONline IDentification) support in the SU field of their INF.

Parameters of the OID and OIR commands:

OID and OIR commands support all contexts (hub-client, client-client). They are allowed in client-client contexts were users to want to avoid their OID/OIR information travelling through hubs. Both clients and hubs MAY provide each other with OID/OIR information.

This extension will add "ADCS transfers are required" as error code in STA. A client that has chosen to only allow encrypted transfers (with ADCS) should send this when a user without ADCS support tries to initiate a connection (via CTM/RCM etc).

This extension will increase searching capability in BASE. The extension also imply that searching in partial file lists are now easier.

Signal ASCH in the INF’s SU field.

The SCH command is extended to request a response (STA) indicating how many search results were sent. This will allow clients to indicate that all search results have been received (and can aptly indicate as such to the user). STA severity 0 and code 00 should be used.

Additional SCH fields:

MT field:

PP field:

RE field:

Additional RES fields:

Additional STA fields:

This extension adds a Date attribute to files and directories in a file list. The attribute is the last modified date of the file or directory. Time specified is seconds since the Unix epoch.

Implementations should be conservative when it includes the Date attribute. Only including the attribute in partial file lists can decrease overall network load requirements.

The following changes are done to the file list XML schema:

A new attribute is defined:

The attribute is then referenced in the File and Directory element:

This extension adds a Size attribute to directories in a file list. The attribute is the size of the directory as indicated in a RES.

Implementations should be conservative when it includes the Size attribute.

This attribute only makes sense in partial file lists (as it can be calculated in full lists).

The following change is done to the file list XML schema:

The Size attribute from BASE is referenced in the Directory element:

This extension adds a Children attribute to directories in a file list. The attribute indicates whether there are additional sub-directories.

Value 1 indicate that there are children.

This attribute only makes sense in partial file lists.

The following changes are done to the file list XML schema:

A new attribute is defined:

The attribute is then referenced in the Directory element:

The info in the current GET command does not permit displaying relative upload progress for the uploading party (for the whole file).

To address this, this extension will add an additional field to the GET command for current downloaded (and verified) bytes before the request has been sent. While still not entirely accurate with this information, the uploader can see how much of the file the requesting party actually has instead of either assuming that the requester has the file up to the start position of the request or being forced to only show the progress of the currently requested part of the file. There is potentially a slight delay in the reporting of this info in scenarios where more than one segment of a file is simultaneously requested (by the downloader) and the uploader still lacks information about how many other sources the file is being downloaded from.

The goal of this extension is to enhance the redirect capabilities provided by BASE by adding support for two new redirect operations and a way for hubs and clients to communicate accepted redirect protocols. It also attempts to make the concept of redirects more unambiguous and less implementation defined. The two new types of redirects added are a multiple choice redirect and so-called permanent redirect as outlined below. These new redirect types are not necessarily mutually exclusive.