Building for Blue Mars
From Blue Mars Developer Guidebook
This page is a general introduction to the concepts of making content for Blue Mars. It starts with a description of what happens in the client software, then gives a general workflow to creating files for the client software. More detailed information and links to other tutorials follow the general workflow.
The Blue Mars platform consists of 4 main parts: The Client software, the Developer toolkit, the website, and the host servers. All of the platform is in open beta development as of September 2010 and is approximately 75% complete at this time. Therefore any information and features described here are liable to change. Check that any page you read is up to date.
The Client Software
The software that players download and install is called the Client. This comes from the Server/Client relationship between the player software and the Avatar Reality host servers which they communicate with. Knowing how players will see and interact with what you build is helpful, so see the Player Guide for more information on using the Client. This section covers how the Client works.
When players start Blue Mars using the desktop icon, it runs the BlueMars.exe program located in their install directory (usually C:\Program Files\Blue Mars\bin32). The client is based on the CryEngine 2 graphics engine made by Crytek (makers of the Crysis game series). The core graphics engine has been modified and extended to add virtual world features, and continues to be developed during the beta period.
In addition to the BlueMars.exe program itself, the install package includes additional items:
The unique data generally is in two or three files:
This file structure discussion brings up two important points about developing for Blue Mars: (1) Path names may not have spaces in them, use underscores instead, and (2) File locations matter a lot, since things are located by their file path under the root Game directory. If a file is moved, the software will not know where to find it, and you will get an error. The developer directory, which is separate from the user client directory, mostly mirrors the same directory structure.
As other cities get downloaded by players, they are placed in added folders under the Game directory. Using the main common.pak and company common.pak saves on file space, since only what is unique to a city has to be downloaded for each added city. When a city is updated, one or more of it's component files may be downloaded again, or a "patch" file with added items is downloaded.
- \Game\Config Folder - This contains "config" files with the .cfg file extension. These are simple text files containing game settings. The format is one Console Variable (CVar) entry per line. These are identical to the console window commands that can be entered in the City Editor when editing a city. The current settings are recorded in My Documents\My Games\BlueMars\game.cfg, which gets generated/modified each time the game runs using the config folder and in-game options settings.
- \Game\Objects Folder - This contains individual content assets, such as avatar clothing, that are not tied to a specific city. The folder starts out empty, and items get streamed to players as needed when they are online. They are also in the form of .pak files. Barring a re-install, both city files and object files get saved to disk and should only have to be downloaded once.
What Happens when Blue Mars runs
When you start the program, you log into your account, and then are presented with a list of cities in the Places Browser. Assuming that city has been downloaded already, when you select one, the level.pak terrain data and city specific .pak asset data are loaded so you can see your surroundings. In turn, any company common and main game common assets that are visible from the starting point are also loaded. All of these are sent to the graphics card of the player's PC. The graphics engine loads various shaders into the graphics card. Shaders are the software that tells how to convert the input 3D models and textures into the finished 3D view on the screen. As the game runs, the graphics card will constantly be sent assets to display, and possibly discarding assets if it's memory gets full.
Once the static city items in view are loaded, then your Avatar gets loaded, along with whatever clothing it was wearing, and the default "idle" animations start running. When you walk or sit or do any other action, another animation is played. As you move around, the graphics engine will constantly re-calculate what is in view to create new frames. Normal playing speed is in the range of 10-50 frames per second, but it is highly dependent on the speed of the graphics card and window size chosen. It is somewhat dependent on the game settings for graphics quality.
When new items such as shop interiors, other avatars wearing new clothes, etc come into view, then those must be streamed to your PC in order for you to see it. An icon will appear on the top left corner when this happens, and it may cause a momentary pause in the game until it finishes. If the scene changes dramatically, such as when you teleport, or turn to face the other way, new data may need to be loaded from hard drive to graphics card, and cause a momentary pause.
Everything in Blue Mars is ultimately made of triangles with a texture (image) applied to it's surface. A typical object may have anywhere from hundreds to thousands of triangles that make up it's shape. The reason for using triangles, is that is the simplest shape that has an area. The shader chosen for that part of the object will take the input texture, determine what part of the texture to put on that particular triangle, apply shading to the color (hence the general name of shader) depending on the angle of the triangle to whatever light sources are around, and apply whatever other effects are part of that specific shader. As a given scene (the part of the city visible in the 3D view) may have millions of triangles in view, the graphics card simply repeats the shading process many many times. Modern graphics cards have hundreds of identical circuits doing the same thing (processing triangles) in parallel, so it gets done very fast, but there is still a limit to what any given card can do. The more triangles in the scene, the fewer frames can be completed in one second. If the scene is too complex, the frame rate will drop to unacceptable levels for players.
The hardware limits of graphics cards speed, hard disk space, and internet speeds leads to a several limits on what you design. See also Technical Limitations
- (1) The total complexity of all the items in view should stay below certain limits. This includes triangle count, number of lights, number of submaterials, etc. Both the relatively static content of a city (terrain, trees, buildings, etc), plus variable content such as shop interiors, avatars and their clothes, have to be considered. More discussion on limits can be found on this Developer Forum Thread.
- (2) Individual items have complexity limits to keep overall scene complexity and download times acceptable, and
- (3) City total file sizes are limited to 1 GB per city.
What You Need for Building:
Developers - Are individuals or companies who create content and assets to be used within the Blue Mars virtual world platform. In other words, that's you the reader.
Developer Account - Is a distinct account type from Player Accounts. It is required to have a developer account to upload assets, own cities, and cash out earnings. Developer accounts automatically create a player account if you do not already have one and Developer Accounts have separate game money balances from player accounts, therefore developers automatically have two balances. You can register for a developer account on the Developer Registration Page
Developer Toolkit - Contains the set of eight Blue Mars Sandbox Editors which are used to convert 3D models to the native format, display them with the CryEngine graphics engine, set up materials and textures, and place objects in the 3D environment. It also contains a developer version of the Client Software so you can test what you make. You can download the current City Developer Toolkit from the Tool Download page. The toolkit installs in a separate directory from the Client software, but it follows the same directory structure. A developer account is not required to download the toolkit, but is for uploading files to Blue Mars.
Sample Data (Optional) - Contains a number of complete cities in editable form. These can be examined to see how they are built, or assets from these cities copied or modified for your own projects. It is also available on the Tool Download page.
Third Party Software - In addition to the Developer Toolkit, you will need a 3D modeling program to create objects from scratch or convert/modify existing models for use in Blue Mars, and an image editing or file conversion program such as Photoshop to prepare textures in the required formats. The Third Party Software Tools page has a partial list of programs to use. If you are only assembling items made by other people into a block or city build, you may not need these programs.
The General Workflow:
This is the general workflow to creating 3D objects such as clothing, furniture, and avatar bodies. Other assets such as scripts, animations, and sounds follow a different process:
(1) Download and install software - Prior to creating content, obtain whatever 3D, image, and other software you will use, and the Blue Mars Developer Toolkit, and install all of them. You will need several GB of hard drive space for a full install + whatever content you make.
(2) Prepare Model & UV Map - This is done in a 3D Modeling program of your choice.
(3) Prepare Textures - This is done in an image editing program such as Photoshop, or in the case of textures which are close enough, simply converted to the Direct Draw Surface .dds final format.
(4) Convert Model - This is done with either the Item, Cloth, Furniture, or Body Editors.
(5) Set up Materials and Textures - This is done using the Material Editor window of the appropriate Blue Mars Editor program.
What happens at this point depends on how the item will get used either:
(6) Pack and Upload Item - The respective Editor is used to pack the item into a .zip file, tested with the developer version of the client software, then uploaded to your developer page online.
(7) Set up a City - Create the editable city files using the City Editor.
(8) Use Items in Block or City - In parallel with (7) add your 3d objects to locations within a block or city using the respective Editors.
(9) Upload City - Using the City Editor, save to game format, then test locally, and if good export and upload files to Avatar Reality.
(10) QA Review and Release - This is done by the Avatar Reality staff, and generally takes 1-2 business days.
The Process In More Detail
(1) Download and Install Software
The Blue Mars BETA Client
- You will most likely want to install the player client software if you have not done so already. That can be obtained from the Client Download page.
- Download the current version of the Developer Toolkit from the Tool Download page. This should normally be installed on your C:\Program Files (x86)\ directory at the same level, but not in the same location as the client software, which would cause file conflict. There are two licenses shown during installation, one from Avatar Reality, and one from Crytek, whose graphics engine and core Editor software we use. See the Developer Toolkit page for more details.
- Install your chosen 3D Modeling, image editing, and other software, and any pre-made content. Certain programs are "supported" ie 3ds Max, Maya, and Photoshop. These have plugins to export to Blue Mars formats and the Avatar Reality staff have tested how they work. Other programs can and are being used for content development, but there is simply not enough staff time to test and adapt to the multitude of available programs. It is your choice what software to use, but support is obviously a factor to consider.
The key requirement is to be able to save in the correct Collada .dae file format, or in a case of a few programs, to export directly to the final Crytek Geometry File (.cgf) format. Collada format files are later converted using one of the Blue Mars Editors to .cgf. There are specific technical requirements on how the model is set up, in order to correctly import and work in Blue Mars. This will vary according to which 3D program you use.
- All objects in Blue Mars, including the avatars, are 3D models, but the Blue Mars editors have very very limited ability to create models. So the first step is to make the model in some 3D program. The native format is "Crytek Geometry File" or .cgf, which a few programs can export. The editors can convert from COLLADA .dae format, which many more can export.
(3) Prepare Textures
The .dds format contains premade smaller versions of the texture, each being half the size of the previous one. As an object gets closer or farther in the game 3D view, the correct texture size is used. Having the various sizes preloaded saves time. Most source images in other formats only have the largest size, hence the need to convert them first.
- Texture maps need to end up as "DirectDraw Surface" or .dds files, which is part of the Microsoft DirectX system. Some 2D graphics programs like Photoshop can export .dds, and the editors will accept TIFF files and convert. More programs can save in .tif format.
(4) Convert Model
The model and image files are placed in a folder under \Game\Objects\MyData\ and converted by running one of the editors. Generally it's best to use a separate folder for each item. The file structure below MyData is however you choose to arrange it.
- If you are making a cube, like the default object in Second Life, then you would place your model file and texture file(s) together in a folder in
\Blue Mars City Developer Tools\Game\Objects\MyData directory, then open the "Item Editor" program.
- In the Item Editor you run a conversion if needed from .dae to .cgf, and display the cube.
(5) Set Up Materials and Textures
All the Blue Mars Editors use the same Cryengine 2 graphics engine as the game, and so display things more or less as they will be seen online. No 3D modeling program, however, uses the full graphics engine. So while partial set up of the materials and textures can be done in your 3D program, the final setup must be done in one of the Editors. All the Editors use a "Material Editor" window for that setup. A texture map is an image file, while a material specifies what texture maps to use, and what graphics engine shaders and settings to apply.
Material settings are recorded in a Material File with .mtl extension. This is an XML format text file which can be edited directly with any text editor, but the Material Editor window is easier to use, and shows changes immediately in the 3D viewport of the editor. One object can have only one material file at a time, but each material file can have multiple submaterials using different textures and settings. Thus a complete item consists of a .cgf model file, .mtl material file, and one or more .dds texture files.
- Then you open the "Material Editor" window and assign the texture map(s), adjust the repeats, rotation, etc. This is similar to the Second Life Edit window, Textures tab. Since no 3D program uses the CryEngine 2 graphics, the material settings have to be done in one of the Editors, which do use the same graphics as the Blue Mars game.
- Now that you have your object set up, you can do two things with it. Pack up the files and upload the item to your Blue Mars developer account "MyPage" or use the item in the City Editor or Block Editor to build with. For the first that is if you want the item in your personal inventory or want to sell it in-world. For the second that is if you want to have it eventually be part of a city.
(6) Pack and Upload Item
Individual items such as clothing, furniture, or avatar skins are "packed" into an itemdata.zip file, tested using the Developer Client, and then uploaded to your Developer Account Page. You can then have it for personal use, or sell copies to other players. The items are downloaded to players' Objects folders mentioned above, as an encrypted .pak file. Anything in the same folder will get packed together, which is why it's generally good to use separate folders for each item.
(7) Set up a City
Besides the 3D objects discussed in the next section, a city contains a number of other things:
- Terrain map
- Custom Settings
This includes setting the terrain size, water depth, and max height, terrain texture layers, environment settings, and vegetation groups. Then either import or paint height map, terrain layers, and vegetation. Next step is setting up blocks within the city, and exporting them so that other people can build their parts of the overall city.
- Blocks are part of a city, similar to land parcels in Second Life. When a block is created, the City Developer exports a block file and sends it to another peson
to work on. Then they send it back and merge it into the city files. The latest city files are exported to share with other people or uploaded to the Blue Mars servers. For a small project that one person is doing, you don't need to create any blocks, just build everything with the City Editor. But for larger projects the blocks let many people work on different parts.
- Scripts use the LUA language, and are edited and tested in the City Editor.
(8) Use Items in Block or City
Items you or other people have made can be used in a Block File (part of a city), or City File. In this case the various files will get copied to the appropriate block or city folder. In the case of a block, the Block Editor is used to pack and export a .b2c file, which gets sent to the city developer. This gets merged by the city developer with other blocks, and their own city level contents to produce a single city file set.
- For simple building, I have a "prims and textures" kit, where I have basic shapes (boxes, cylinders, cones, etc) already made, and all you have to do is drag them into the perspective view and apply the texture. This is almost the same as dragging an object from your inventory to the ground in SL and applying the texture to it.
(9) Upload City
City files are first exported to game engine format, then packed for upload into two .cty files. One is the company level common data, and the other is the city specific data. Again, whatever is in the directory folders will get packed up, so you should try to include what is needed, remove anything not being used, and make sure files are where they are referenced or the software will not be able to find it. That generally means don't move a file after you have used it. The specific details of how to upload city files are provided when you purchase city hosting. This deters unauthorized people from uploading those files. Since development is done locally on your PC, you can build a city first, and then wait to purchase a hosting plan until just before you are ready to upload it.
Revisions involve saving and exporting the full city files again. The Blue Mars build staff will generate patch or new .pak files from your data for players to download.
- Cities can be logged into and run in "Developer Mode" for testing purposes. The difference between Developer Mode and normal game mode is where you get the city files from, and which directory you put those files in and run the bluemars.exe from. Once you are ready to have your city open to the world, then you upload the city files to their servers and it will then be listed in their directory of cities. Also if you want to sell things it needs to be set up on their servers.
(10) Quality Assurance Review
All uploads (items and cities) are reviewed by the Avatar Reality QA staff for technical problems and mature content. This takes approximately 1-2 working days. Once done, the items will be "released" and available to be used, or downloaded. Cities generally come as partial updates, rather than a complete download once you have the initial version.