Difference between revisions of "SOE Protocol Breakdown"

Revision as of 15:56, 3 March 2007

SOE's Protocol is a systematic way to enhance the UDP protocol to make it more reliable, and also save bandwidth by merging and grouping packets. It can be combined in various ways, or not be used at all.

The format of the SOE Protocol varies depending on what type of operation it is. SOE Protocol can function as a stand alone packet, or as a Header to SWG Packets. What is true about all of the SOE Protocol is that the beginning contains 2 bytes known as the SOE Opcode. Majority also contain a footer. Assume all packets have footers unless otherwise mentioned. The table of opcodes are as follows:

Note An SOE Opcode is stored as an unsigned short (uint16), and is in net byte order.

(SOE Opcode)"in hex"	(Meaning) 		(       (ENUM_NAME) "used in code"

00 01		Session Request				(SOE_SESSION_REQUEST)
00 02		Session Response			(SOE_SESSION_REPLY)
00 03		Multiple SOE Protocols			(SOE_MULTI_SOE)
00 04		Not Used				(SOE_NOT_USED)
00 05		Disconnect				(SOE_DISCONNECT)
00 06		Keep Alive				(SOE_PING)
00 07 		Client Network Status Update		(SOE_NET_STATUS_REQ)
00 08 		Server Network Status Update		(SOE_NET_STATUS_RES)

00 09 		Channel 0 - Data Packet			(SOE_CHL_DATA_A)
00 0A		Channel 1 - Data Packet			(SOE_CHL_DATA_B)
00 0B		Channel 2 - Data Packet			(SOE_CHL_DATA_C)
00 0C		Channel 3 - Data Packet			(SOE_CHL_DATA_D)

00 0D		Channel 0 - Fragmented Data Packet	(SOE_DATA_FRAG_A)
00 0E		Channel 1 - Fragmented Data Packet	(SOE_DATA_FRAG_B)
00 0F		Channel 2 - Fragmented Data Packet	(SOE_DATA_FRAG_C)
00 10		Channel 3 - Fragmented Data Packet	(SOE_DATA_FRAG_D)

00 11		Channel 0 - Out of Order Packet		(SOE_OUT_ORDER_PKT_A)
00 12		Channel 1 - Out of Order Packet		(SOE_OUT_ORDER_PKT_B)
00 13		Channel 2 - Out of Order Packet		(SOE_OUT_ORDER_PKT_C)
00 14		Channel 3 - Out of Order Packet		(SOE_OUT_ORDER_PKT_D)

00 15		Channel 0 - Acknowledge Packet		(SOE_ACK_A)
00 16		Channel 1 - Acknowledge Packet		(SOE_ACK_B)
00 17		Channel 2 - Acknowledge Packet		(SOE_ACK_C)
00 18		Channel 3 - Acknowledge Packet		(SOE_ACK_D)

00 19		Channel 0 - Multi SWG Packets		(SOE_MULTI_SWG_A)
00 1A		Channel 1 - Multi SWG Packets		(SOE_MULTI_SWG_B)
00 1B		Channel 2 - Multi SWG Packets		(SOE_MULTI_SWG_C)
00 1C		Channel 3 - Multi SWG Packets		(SOE_MULTI_SWG_D)

00 1D		*Serious Error Acknowledge		(SOE_FATAL_ERR)
00 1E		*Serious Error Reply			(SOE_FATAL_ERR_REP)

Note These functions are seen on very malformed packets. Very rare occurance and the client will usually disconnect. It's inclusion in the documentation is for completion ONLY. SWGEmu will disconnect automatically when reciving a Error Acknowledge, and it will NOT send any, rather just disconnect itself.

SOE has their protocol premade to allow various channels of data transmission. However, no game to this date utilizes the multiple channels. SWG will only be using channel 0 and it will be assumed throughout the rest of the packet documentation.

The following data is of packet breakdowns and explainations on each type. See the article "Packet Breakdown Format" for information on the specific used for documentation.

Packet Type -- 00 01

00 01  -  Session Request (Client -> Server)
SOE Opcode	- Net Byte SHORT	
CRC Length	- Net BYte INT
Connection ID	- Net Byte INT
ClientUDPSize	- Net Byte INT	

• SOE Opcode is just the opcode number to identify the packet.
• CRC Length is the amount of length of the CRC checksum to append at the end of a packet. SWG uses 2 bytes.
• Note CRC32 checksums are 32bit, making them 4 bytes, but only 2 are appended.
• Connection ID is some type of identification used for the connection. Only other time seen is during a disconnect.
• ClientUDPSize is the maximum size allocated for the client's UDP packet buffer. No packet is allowed to exceed this size. If it is larger, it must be fragmented. This size is equal to 496 bytes.

Note This opcode DOES NOT have a footer.

Packet Samples -- 00 01

0x0001

00 00 00 02 27 38 24 61 00 00 01 F0

....'8$a....

0x0001

00 00 00 02 3C D6 0F BF 00 00 01 F0

....<.......

Packet Type -- 00 02

SOE Opcode	- Net Byte SHORT
Connection ID	- Net Byte INT
CRCSeed		- Net Byte INT
CRCLength	- BYTE
UNKNOWN		- SHORT
ServerUDPSize	- Net Byte INT

• Connection ID is replied using the same ID sent by the Session Request.
• CRCSeed is a seed value used for the calculation of the CRC32 Checksum.
• CRCLength
• UNKNOWN is an unknown value that is constant and equal to 0x0104 (260).
• ServerUDPSize is the maximum size allocated for the server's UDP packet buffer. No packet is allowed to exceed this size. So far the client has not sent anything large enough to fill this, or be fragmented. This size is equal to 496 bytes.

Note This opcode DOES NOT have a footer.

Packet Sample -- 00 02

0x0002

27 38 24 61 37 E4 BF 63 02 01 04 00 00 01 F0

'8$a7..c.......

0x0002

3C D6 0F BF 3B BF 9B D5 02 01 04 00 00 01 F0

<...;..........

Packet Type -- 00 03

SOE Opcode	- Net Byte SHORT
Data Size	- (VARIOUS)
DATA		- (VARIOUS)
Data Size	- (VARIOUS)
DATA		- (VARIOUS)
...

These packets allow the combination of 2 or more SOE Protocol packets or stand alone SWG Packets with SOE Protocol packets.

(SOE Protocol)((SOE Protocol)(SWG Packet)) or (SOE Protocol)(SWG Packet)

or as a better example:

(Multiple SOE Proto)(Ackowledge)(Data) or (Multiple SOE Proto)(Acknowledge)(SWG Packet) or (Multiple SOE Proto)(Acknowledge)(Acknowledge)

Each seperate packet is preceded by it's size. So a better example would be: (Multiple SOE Proto)(size1)(Acknowlede)(size2)(Data)

so a very basic hexadecimal version would be:

(00 03)  (04)  (00 15 00 00)  (0A)  (00 09 00 01 01 00 FF FF FF FF)
 multi   size   acknowledge   size              data

• NOTE: The Size field will never exceed 255.

Packet Sample -- 00 03

0x0003

06 01 00 AF F9 6C A1 1C 0A 00 08 F8 24 1B A8 1C 
95 AF 22 00 00 00 A2 54 40 01 2D 55 45 02 00 00 
00 D1 00 00 1C 0A 00 08 F8 24 1B A2 04 95 AF 22 
00 00 00 70 54 40 01 47 55 55 02 00 00 09 EB 00 
00 1C 0A 00 08 F8 24 1B AC 04 95 AF 22 00 00 00 
4E 54 40 01 5A 55 2D 01 00 00 0A EF 00 00 1C 0A 
00 08 F8 24 1B 70 04 95 AF 22 00 00 00 48 54 40 
01 16 55 17 02 00 00 0A 15 00 00 1C 0A 00 08 F8 
24 1B DB FA 94 AF 22 00 00 00 72 54 40 01 5E 55 
06 02 00 00 0A 30 00 00 1C 0A 00 08 F8 24 1B CA 
FD 94 AF 22 00 00 00 13 54 40 01 86 56 C3 06 00 
00 00 FC 00 00 1C 0A 00 08 F8 24 1B C0 FD 94 AF 
22 00 00 00 11 54 40 01 87 56 96 06 00 00 00 FD 
00 00 1C 0A 00 08 F8 24 1B 8E FD 94 AF 22 00 00 
00 55 53 40 01 8C 57 FC 06 00 00 02 D2 00 00 1C 
0A 00 08 F8 24 1B 70 FD 94 AF 22 00 00 00 61 53 
40 01 BF 57 26 07 00 00 02 D2 00 00 1C 0A 00 08 
F8 24 1B AC FD 94 AF 22 00 00 00 0F 54 40 01 86 
56 AE 06 00 00 00 FD 00 00 C1 00 09 19 64 00 19 
1C 0A 00 08 F8 24 1B 7A 04 95 AF 22 00 00 00 8D 
54 41 01 B8 54 EA 04 00 00 00 02 00 00 1C 05 00 
46 5E CE 80 1B 00 00 00 31 01 00 00 C9 1C 95 AF 
22 00 00 00 00 00 00 00 0E 01 46 05 00 46 5E CE 
80 1B 00 00 00 CC 00 00 00 AC 04 95 AF 22 00 00 
00 00 00 00 00 C0 B1 16 F3 AC 04 95 AF 22 00 00 
00 B2 04 95 AF 22 00 00 00 00 10 00 B7 2C 10 73 
00 01 00 C9 1C 95 AF 22 00 00 00 0E 01 00 00 4D 
0A 1C 0A 00 08 F8 24 1B B6 FD 94 AF 22 00 00 00 
11 54 40 01 86 56 92 06 00 00 00 FD 00 00 1C 0A 
00 08 F8 24 1B 0F F7 94 AF 22 00 00 00 C7 53 40 
01 0E 56 5A 08 00 00 02 29 00 00

.....l......$....."[email protected]...........
.$....."[email protected]............$....."...
[email protected]............$.p..."[email protected].....
........$....."...rT@.^U.....0.......$..
..."[email protected].............$....."....T@.
.V.............$....."[email protected].........
....$.p..."[email protected]&............$....."
[email protected].......$.z..."....
TA..T...........F^......1.......".......
..F..F^.............."..............."..
.....".......,.s.......".......M......$.
...."[email protected].............$....."....S@
..VZ....)..

0x0003

04 00 11 01 8E 04 00 11 01 8F

..........

0x0003

04 00 15 0B 1E 9E 00 09 07 9E 00 19 2E 05 00 46 
5E CE 80 23 00 00 00 16 01 00 00 13 6C 35 E3 16 
00 00 00 00 00 00 00 80 00 00 00 02 B7 A8 D6 9B 
FC 19 AE 1A 00 00 00 00 00 00 00 22 05 00 46 5E 
CE 80 83 00 00 00 26 01 00 00 13 6C 35 E3 16 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 22 05 
00 46 5E CE 80 83 00 00 00 26 01 00 00 13 6C 35 
E3 16 00 00 00 00 00 00 00 DF B5 1A AE 1A 00 00 
00 22 05 00 46 5E CE 80 83 00 00 00 26 01 00 00 
13 6C 35 E3 16 00 00 00 00 00 00 00 9B FC 19 AE 
1A 00 00 00 4F 05 00 46 5E CE 80 21 00 00 00 F1 
00 00 00 13 6C 35 E3 16 00 00 00 00 00 00 00 81 
49 E9 85 A2 01 00 00 EF B4 1A AE 1A 00 00 00 00 
00 00 00 33 FD 68 BD 00 00 00 00 E5 95 7F 3F D8 
AE 3E 3F 33 33 33 3F CA 56 49 40 00 00 00 41 00 
00 00 00 00 F0 00 09 07 9F 00 19 2E 05 00 46 5E 
CE 80 23 00 00 00 16 01 00 00 13 6C 35 E3 16 00 
00 00 00 00 00 00 80 00 00 00 02 B7 A8 D6 9B FC 
19 AE 1A 00 00 00 00 00 00 00 22 05 00 46 5E CE 
80 83 00 00 00 26 01 00 00 13 6C 35 E3 16 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 22 05 00 
46 5E CE 80 83 00 00 00 26 01 00 00 13 6C 35 E3 
16 00 00 00 00 00 00 00 9B FC 19 AE 1A 00 00 00 
2E 05 00 46 5E CE 80 23 00 00 00 16 01 00 00 13 
6C 35 E3 16 00 00 00 00 00 00 00 80 00 00 00 02 
B7 A8 D6 9B FC 19 AE 1A 00 00 00 00 00 00 00 22 
05 00 46 5E CE 80 83 00 00 00 26 01 00 00 13 6C 
35 E3 16 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 22 05 00 46 5E CE 80 83 00 00 00 26 01 00 
00 13 6C 35 E3 16 00 00 00 00 00 00 00 9B FC 19 
AE 1A 00 00 00

...............F^..#........l5..........
..................."..F^......&....l5...
.............."..F^......&....l5........
........."..F^......&....l5.............
....O..F^..!........l5..........I.......
...........3.h.......??..>[email protected].
..............F^..#........l5...........
.................."..F^......&....l5....
............."..F^......&....l5.........
...........F^..#........l5..............
..............."..F^......&....l5.......
.........."..F^......&....l5............
.....

0x0003

04 00 15 02 0B 04 00 15 02 0F

..........

Packet Type -- 00 04

SOE Opcode	- Net Byte SHORT
Connection ID	- Net Byte INT
Reason ID	- Net Byte SHORT

• Connection ID is the ID number assigned during the session request. Each client sends its own Connection ID. Reason ID is an ID number that corresponds for a reason for the disconnection.

This packet is sent whenever one side wants to disconnect. It resends the clients specific Connection ID established during the session handshake, and then a reason for disconnection. So far the standard reason ID seen has been 6. Others have been tested and do not have any different effect.

No table correspnding IDs to reasons has been located.

Packet Type -- 00 06

SOE Opcode	- Net Byte SHORT

This packet is simply a keep alive to tell the other side that is still connected, but idle. It only consists of the opcode and a footer.

Packet Samples -- 00 06

0x0006

CB 3A 78

.:x

0x0006

16 01 A7

...

0x0006

C7 23 8F

.#.

0x0006

CB 3A 78

.:x

Packet Type -- 00 07

SOE Opcode	- Net Byte SHORT
Data..

These packets consist of various data and network statistics. Because we do not care about network statistics, we will ignore these. To ignore these we send a special malformed Server Network Status Update.

Note: This packet is usually compressed but is optional.

Packet Samples -- 00 07

0x0001

CA 3A 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 
00 00 00 00 00 01

.:....................................

0x0001

D2 C7 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 
00 00 00 00 00 01 

......................................

Packet Type -- 00 08

SOE Opcode	- Net Byte SHORT
Data..

This packet is supposed to contain the Servers network statistics, however we felt it was a waste of time to reverse these as we do not care about the statistics. Instead we send 38 bytes of 0x00, and then compress the packet, but skip appending a CRC. The client seems to register that it recieved the servers statistics, but notices a bad CRC (or none in our case), and tosses out the packet as malformed. Because it recieved the packet it continues on without erroring.

Note: This packet is usually compressed but is optional.

Packet Type -- 00 09

SOE Opcode	- Net Byte SHORT
Sequence #	- Net Byte SHORT
SWG PACKET..

Sequence # is a sequence number applied to every data packet. It increments in order for each packet and is used to help determine out of order packets. This protocol is used to send an SWG Packet that NEEDS to be recieved by the other end. The reciver is then supposed to respond with a an ACKNOWLEDGE packet that contains the same sequence so the sender knows it has been recieved. This system ensures packets not only get sent correctly, but also stay in order.

After the sequence # comes the encapsulated SWG Packet.

Note* it is possible to combine multiple SWG PACKETs into a single 09. This is done similar to the 0x0003 packet, a multi data example:

(00 09) (sequence) (00 19) (Size1) (SWG PACKET) (Size2) (SWG PACKET)
SOE op.  sequence   multi   size       data       size      data

if the Size byte is FF the following 2 bytes are the size Word Little ended (16bit). Example:

(00 09) (sequence) (00 19) (size byte) (size word) (SWG PACKET) (size byte) (SWG PACKET)
SOE op.  sequence   multi    FF          01 99        data         33          data

In the above the first packet size is 409 Dec.

Use these to combine SWG PACKETs together into 1 data packet/sequence number.

Packet Type -- 00 0D

SOE Opcode	- Net Byte SHORT
Sequence #	- Net Byte SHORT
*FragmentSize	- Net Byte INT
SWG Packet...

• Sequence # is the same type of sequence number described in the "00 09 - Channel 0 Data" section. All the same rules apply.

• FragmentSize is the total size of the combined fragments. This only appears in the FIRST packet of a group of fragmented packets.

Next is just the piece data of the large SWG Packet. Fragments are made when the total UNCOMPRESSED packet exceeds the packet buffer size, which is 496 bytes.

Packet Type -- 00 11

SOE Opcode	- Net Byte SHORT
Sequence #	- Net Byte SHORT

Sequence # is the sequence number of the out of order packet recieved. It will send one for every packet that is out of order, and the ones following.

So if packet sequences recieved are:

00 00
00 01

And then:

00 03

an out of order will be sent for the packet with sequence 00 03.

If the packet sequences recieved are:

00 00
00 01

And then:

00 03
00 04
00 05

An out of order packet will be sent for all 3 of the out of order sequences.

Packet Type -- 00 15

SOE Opcode	- Net Byte SHORT
Sequence #	- Net Byte SHORT

• Sequence # is the sequence number of the packet to acknowledge to the sender that it was recieved correctly.

Every packet sent with a sequence number MUST be acknowledeged.

They do not neccesarily need to be Acknowledeged 1 by 1. For example: If packet sequences recieved are: 00 00 00 01 00 02

An Acknowledge of 00 15 00 02 will cover all the packets up to the sequence 00 02.

The client will stop processing packets after about 4 unacknowledeged packets so frequent acknowledges are needed.

Thanks to Acid1789 for the abundant information on the SOE Protocol!