aMule
	     - follow the white rabbit -


		External Connections
		     Protocol

		   version 2.0

Preface
-------

    EC is under heavy construction, however the protocol itself is considered
    stable and you can rely on. The opcodes and tagnames, tag content formats
    and values are still changing, so if you decide to implement an application
    using aMule EC, you'd better include our ECcodes.h for the values, and
    check the documentations often, or even the code itself (ExternalConn.cpp
    is a good start).



Section 1: Protocol definition
------------------------------

Short description:

    EC protocol consist of two layers: a low-level transmission layer, and
    a high level application layer.


Section 1.1: Transmission layer
-------------------------------

    The transmission layer is completely independent of the application layer,
    and holds only transport-related information.

    The transmission layer actually consists of an uint32 number, referenced
    below as flags, which describes flags for the current transmission session
    (send/receive operation).

    This four-byte value is the only one in the whole protocol, that is
    transmitted LSB first, and zero bytes omitted (therefore an empty
    transmission flags value is sent as 0x20, not 0x20 0x0 0x0 0x0).

    Bit description:

	bit 0:	Compression flag. When set, zlib compression is applied to
		the application layer's data.

	bit 1:	Compressed numbers. When set (presumably on small packets
		that doesn't worth compressing by zlib), all the numbers used
		in the protocol are encoded as a wide char converted to utf-8
		to let some zero bytes not to be sent over the network.

	bit 2:	Has ID. When this flag is set, an uint32 number follows the
		flags, which is the ID of this packet. The response to this
		packet also has to have this ID. The only requirement for the
		ID value is that they should be unique in one session (or at
		least do not repeat for a reasonably long time.)

	bit 3:	Reserved for later use.

	bit 4:	Accepts value present. A client sets this flag and sends
		another uint32 value (encoded as above, LSB first, zero
		bytes omitted), which is a fully constructed flags value,
		bits set meaning that the client can accept those extensions.
		No extensions can be used, until the other side sends an
		accept value for them. It is not defined when this value
		should be send, best is on first transfer, but can be sent
		any time later, even changing the previously announced
		flags.

	bit 5:	Always set to 1, to distinguish from older (pre-rc8) clients.

	bit 6:	Always set to 0, to distinguish from older (pre-rc8) clients.

	bits 7,15,23: Extension flag, means that the next byte of the flags is
		present.

	bits 8-14,16-22,24-32: Reserved for later use.


    Transmission layer example:
    	0x30 0x23 <appdata>	- Client uses no extensions on this packet,
		and indicates that it can accept zlib compression and
		compressed numbers.

    Notes:
	Note 1:	On the "accepts" value, the predefined flags must be set to
		their predefined values, because this can be used as a sort
		of a sanity check.

	Note 2:	Bits marked as "reserved" should always be set to 0.



Section 1.2: Application layer
------------------------------

    Data transmission is done in packets. A packet can be considered as a
    special tag - with no data, no tagLen field, and with the tagCount
    field always present. All numbers part of the application layer are
    transmitted in network byte order, i.e. MSB first.
	A packet contains the following:
	[ec_opcode_t] OPCODE
	[uint16] TAGCOUNT
	    <tags>

    In detail: The opcode means what to to or what the data fields contain.
    Its type is set as ec_opcode_t, which currently is an uint8.
    TagCount is the number of first level tags this packet has. Then are the
    tags themselves.

    A tag consist of:
	[ec_tagname_t] TAGNAME
	[ec_tagtype_t] TAGTYPE
	[ec_taglen_t] TAGLEN
	<[uint16] TAGCOUNT>?
	    <sub-tags>
	    <tag data>

    The ec_tagname_t is defined as an uint16, ec_taglen_t as an uint32 value
    at the moment. ec_tagtype_t is an uint8. 
	TagName tells what it contains (see ECcodes.h for details).
	TagType sends the type of this tag (see ECPacket.h for types)
    TagLen contains the whole length of the tag, including the lengths of the
    possible sub-tags, but without the size of the tagName, tagType and 
	tagLen fields. Actually the lowest bit of the tagname doesn't belong to the 
	tagName itself, so it has to be cleared before checking the name.

    Tags may contain sub-tags to store the information, and a tagCount field
    is present only for these tags. The presence of the tagCount field can
    be tested by checking the lowest bit of the tagName field, when it is
    set, tagCount field present.

    When a tag contains sub-tags, the sub-tags are sent before the tag's own
    data. So, tag data length can be calculated by substracting all sub-tags'
    length from the tagLen value, and the remainder is the data length, if
    non-zero.


Section 2: Data Types
---------------------

    Integer types
    -------------

    Integer types (such as uint8, uint16, uint32) are always transmitted in
    network byte order (MSB first).


    Strings
    -------

    Strings are always UTF-8 strings, with the trailing zero byte included.
    All strings coming from the server are untranslated, but their translations
    are included in amule's translation database (amule.mo).


    Boolean
    -------

    This one is tricky. When reading, the tag's presence means true, and
    false when omitted. When writing, they should always be present -
    if not, it's considered 'unchanged' - and should hold an uint8 value.
    This value determines the boolean value in the standard way, i.e.
    zero means false and non-zero means true.

    Boolean values are mostly used in reading/writing preferences.


    MD5 Hashes
    ----------

    They are always MSB first.


    Floating point numbers
    ----------------------

    Floating point numbers such as 'float' or 'double' types are converted
    to their string representation, and are sent as string. Note, that the
    decimal point is always the '.' (dot) character, independent from the
    current locale.


Section 3: Clarifying things
----------------------------

    If all the above seemed too much technical, just keep on reading. If you
    understood the above at first, you can safely skip this section.

    Have you ever seen an xml file? Do you know how it looks like? Then you
    can safely think of an EC packet as binary xml. Otherwise (hmm, you
    don't know what xml is?) think of it as a tree, it has exactly one root,
    may have many branches and leaves. We'll use the tree example below.

    But before we get to the examples, just some words about the flags (which
    are part of the transmission layer, you remember?): When developing your
    EC application (frontend to aMule, etc), this might be the last thing you
    want to care about, and it's ok. Just keep sending a byte of 0x20 as flags,
    and aMule will never want to use any of the features described in
    Section 1.1. You just have to take of the "accepts" value aMule will send
    in its first reply.

    An now, to the examples. The example packets are real-life EC packets,
    transscripted to textual form for your pleasure :)

    First, let's see a simple but very important packet: authentication to
    aMule. This must be the very first one, otherwise aMule might drop the
    connection.

    EC_OP_AUTH_REQ (0x02)
        +----EC_TAG_CLIENT_NAME (0x06) (optional)	
        +----EC_TAG_PASSWD_HASH (0x04)
        +----EC_TAG_PROTOCOL_VERSION (0x0c)
        +----EC_TAG_CLIENT_VERSION (0x08) (optional)
        +----EC_TAG_VERSION_ID (0x0e) (required for cvs versions, must not be present for release versions)

    Now, what exactly gets transmitted? Here it comes, with comments (all
    numbers are hexadecimal, I omitted the 0x prefix for redability):

    20                                  FLAGS, just stating that we use ECv2
    02                                  EC_OP_AUTH_REQ
      00 05                             Number of children (tags) this packet has
        00 06                           EC_TAG_CLIENT_NAME
		  0?								EC_TAGTYPE_STRING
          00 00 00 09                   Length of this tag (9)
          61 4d 75 6c 65 63 6d 64 00    Contents of the tag: "aMulecmd", with trailing zero included (see String types in Section 2)
        00 08                           EC_TAG_CLIENT_VERSION
		  0?								EC_TAGTYPE_STRING
          00 00 00 04                   Length of this tag (4)
          43 56 53 00                   Content: "CVS"
        00 0c                           EC_TAG_PROTOCOL_VERSION
		  0?								EC_TAGTYPE_UINT??
          00 00 00 02/4/8         Length: 2/4/8 (16/32/64 value follows)
          00? 00? 01 f2            Content: 0x0200 (current protocol version number for cvs)
        00 04                           EC_TAG_PASSWD_HASH
		  0?								EC_TAGTYPE_HASH
          00 00 00 10                   Length (16)
          5d 41 40 2a bc 4b 2a 76       Content: 16 bytes md5sum of EC password
          b9 71 9d 91 10 17 c5 92
        00 0e                           EC_TAG_VERSION_ID
		  0?								EC_TAGTYPE_CUSTOM
          00 00 00 21                   Length: 33
          62 66 39 64 64 32 36 35       Content 33 bytes of the unique EC CVS version ID
          32 36 34 35 31 36 63 39       Remember: this is only for CVS versions, and its
          34 35 38 36 38 66 61 39       size, content-type, anything might change without
          30 38 66 62 37 64 39 38       notice. For release versions this tag MUST NOT be
          00                            present.

    Now, that we constructed a packet, stating that we are "aMulecmd CVS", send
    it to the server and see what it replies. Hopefully the following:

    30                                  FLAGS, stating that the server sends us an 'accepts' flags
    23                                  the 'accepts' flag. Just take care that your program can
                                        handle when it is present, and we can forget about it for now.
    04					EC_OP_AUTH_OK
      00 01                             Number of children (tags) in this packet
        00 76                           EC_TAG_SERVER_VERSION
		  0?								EC_TAGTYPE_STRING
          00 00 00 04                   Length of this tag (4)
          43 56 53 00                   And the contents: "CVS"

    That was easy. Heading for a more complex example: now that we're connected
    to aMule, ask simple stats from core.

    The request:

    EC_OP_STAT_REQ
        +----EC_TAG_DETAIL_LEVEL (with EC_DETAIL_CMD value)

    20                                  FLAGS, as above
    0a                                  EC_OP_STAT_REQ
      00 01                             TagCount: 1
        00 10                           EC_TAG_DETAIL_LEVEL
		0?									EC_TAGTYPE_UINT8
          00 00 00 01                   Length: 1
          00                            0 = EC_DETAIL_CMD

    The reply (assuming core is connected to a server):

    EC_OP_STATS
        +----EC_TAG_STATS_UL_SPEED
        +----EC_TAG_STATS_DL_SPEED
        +----EC_TAG_STATS_UL_SPEED_LIMIT
        +----EC_TAG_STATS_DL_SPEED_LIMIT
        +----EC_TAG_STATS_CURR_UL_COUNT
        +----EC_TAG_STATS_TOTAL_SRC_COUNT
        +----EC_TAG_STATS_CURR_DL_COUNT
        +----EC_TAG_STATS_TOTAL_DL_COUNT
        +----EC_TAG_STATS_UL_QUEUE_LEN
        +----EC_TAG_STATS_BANNED_COUNT
        +----EC_TAG_CONNSTATE
             +----EC_TAG_SERVER
                  +----EC_TAG_SERVER_NAME

    I won't copy here the whole reply packet, only the interesting part:

    20                                  FLAGS
    0c                                  EC_OP_STATS
      00 0b                             Number of (first-level) tags in this packet: 11 (direct children of the packet root)
        00 14 [...]                     EC_TAG_STATS_UL_SPEED
        00 16 [...]                     EC_TAG_STATS_DL_SPEED
        00 18 [...]                     EC_TAG_STATS_UL_SPEED_LIMIT
        00 1a [...]                     EC_TAG_STATS_DL_SPEED_LIMIT
        00 1c [...]                     EC_TAG_STATS_CURR_UL_COUNT
        00 22 [...]                     EC_TAG_STATS_TOTAL_SRC_COUNT
        00 1e [...]                     EC_TAG_STATS_CURR_DL_COUNT
        00 20 [...]                     EC_TAG_STATS_TOTAL_DL_COUNT
        00 26 [...]                     EC_TAG_STATS_UL_QUEUE_LEN
        00 24 [...]                     EC_TAG_STATS_BANNED_COUNT
        And now the interesing part:
        00 13                           EC_TAG_CONNSTATE. Note, that all tagnames are even numbers, so when
                                        we find an odd number, the true tag name is <found>-1. EC_TAG_CONNSTATE = 0x0012,
                                        and 0x0013 - 1 = 0x0012, so it is really this one. The tagname being an odd number
                                        means that this tag has child(ren) tags (see tree above), and also that it has a
                                        tagcount field too.
		  0?							EC_TAGTYPE_UINT32
          00 00 00 26                   TagLen: 38 (own content length + length of children (with headers))
          00 01                         TagCount: 1 (one child tag exists, that is a direct child of this tag)
            00 61                       EC_TAG_SERVER (has children)
			  0?							EC_TAGTYPE_IPV4
              00 00 00 1a               TagLen: 27 (own content (6) + length of child (content: 14 + header: 7))
              00 01                     TagCount: 1
                00 62                   EC_TAG_SERVER_NAME
				  0?							EC_TAGTYPE_STRING
                  00 00 00 0e           TagLen: 14
                  52 61 7a 6f 72 62 61 63  Content of the EC_TAG_SERVER_NAME tag: "Razorback 2.0"
                  6b 20 32 2e 30 00
              c3 f5 f4 f3 12 35         Content of the EC_TAG_SERVER tag: Server IP:Port (195.245.244.243:4661)
          90 cc 83 52                   Content of the EC_TAG_CONNSTATE tag: Current UserID

    Hopefully these examples helped enlightening the nature of OpCodes, Tags,
    nested tags.