Diffing Unreal Assets

Originally posted on the Unreal Engine blog.

So, source control is awesome, and you should be using it! One of the benefits of source control is being able to look at previous versions of files and compare or ‘diff’ them to see how things have changed. However, regular diff tools don’t work very well on binary files like Unreal .uasset packages. We’ve got you covered with support for exporting a text version of assets that can be diffed using your preferred tool, as well as a built-in specialized diff tool for Blueprints.

Setup

You’ll need to store your project in a supported source control system for the built-in tools to work. The editor currently has source control plugins for both Perforce and SVN. Perforce is what we use internally at Epic, and it is now free for up to 20 users (and 20 workspaces; you’ll probably end up using 2 or 3 per human user). SVN is totally free and there are a number of cloud SVN providers if you don’t want to host your own server.

Note: SVN binaries were not included in the 4.0 release, check out this thread for more details.

You can tell if your editor is currently connected to source control by the little icon on the top right of the main frame, next to the [Enter console command] prompt. If the icon is green, you are already connected, but if it shows up as a red no sign then click on it to enter your server settings.

How to enable source control
Source control enabled Enabling source control

Next, you need to tell the editor where it can find your favorite text diff tool in Editor Preferences. It defaults to p4merge, but almost any diff tool should work (e.g., Beyond Compare, Araxis Merge, Tortoise Merge, etc…)

Picking your preferred diff tool
Picking your preferred diff tool

Diffing assets

Once you are connected, you’ll see additional options in the context menu for Assets, such as ‘Check Out’, ‘History’, and ‘Diff Against Depot’. Any asset can be diffed, and the default handler will export the asset to a readable text format and launch your preferred text diffing tool (some assets have a more sophisticated viewer). Comparing your current version against depot before checking in is a useful habit to form; it will help you write better checkin comments and can also help catch changes you made to test something but didn’t intend to keep.

Source control context menu commands
Source control context menu commands

History gives you a full list of the previous checkins and their descriptions, and you can diff any two arbitrary revisions from the history window (select two revisions and right-click). This can be extremely useful when trying to track down a regression (be kind to future you and write good checkin comments).

Revision history display
Revision history display

Diffing Blueprints

Blueprints are a much more visual medium and a textual diff wouldn’t be very helpful. The editor has a built-in diffing tool for blueprints where you can look at differences in each graph or diff the default values. We’ll talk about this in more detail in a future blog post, but here’s a picture of the tool

Blueprint Diff Tool
Blueprint Diff Tool

Unreal Property System (Reflection)

Originally posted on the Unreal Engine blog.

Reflection is the ability of a program to examine itself at runtime. This is hugely useful and is a foundational technology of the Unreal engine, powering many systems such as detail panels in the editor, serialization, garbage collection, network replication, and Blueprint/C++ communication. However, C++ doesn’t natively support any form of reflection, so Unreal has its own system to harvest, query, and manipulate information about C++ classes, structs, functions, member variables, and enumerations. We typically refer to reflection as the property system since reflection is also a graphics term.

The reflection system is opt-in. You need to annotate any types or properties that you want to be visible to the reflection system, and Unreal Header Tool (UHT) will harvest that information when you compile your project.

Markup

To mark a header as containing reflected types, add a special include at the top of the file. This lets UHT know that they should consider this file, and it’s also required for the implementation of the system (see the ‘A peek behind the curtain’ section for more information).

#include "FileName.generated.h"

You can now use UENUM(), UCLASS(), USTRUCT(), UFUNCTION(), and UPROPERTY() to annotate different types and member variables in the header. Each of these macros goes before the type or member declaration, and can contain additional specifier keywords. Let’s take a look at a real world example (from StrategyGame):

// Base class for mobile units (soldiers)
 
#include "StrategyTypes.h"
#include "StrategyChar.generated.h"
 
UCLASS(Abstract)
class AStrategyChar : public ACharacter, public IStrategyTeamInterface
{
	GENERATED_UCLASS_BODY()
 
	/** How many resources this pawn is worth when it dies. */
	UPROPERTY(EditAnywhere, Category=Pawn)
	int32 ResourcesToGather;
 
	/** set attachment for weapon slot */
	UFUNCTION(BlueprintCallable, Category=Attachment)
	void SetWeaponAttachment(class UStrategyAttachment* Weapon);
 
	UFUNCTION(BlueprintCallable, Category=Attachment)
	bool IsWeaponAttached();
 
protected:
	/** melee anim */
	UPROPERTY(EditDefaultsOnly, Category=Pawn)
	UAnimMontage* MeleeAnim;
 
	/** Armor attachment slot */
	UPROPERTY()
	UStrategyAttachment* ArmorSlot;
 
	/** team number */
	uint8 MyTeamNum;
 
	[more code omitted]
};

This header declares a new class called AStrategyChar deriving from ACharacter. It uses UCLASS() to indicate it is reflected, which is also paired with a macro GENERATED_UCLASS_BODY() inside of the C++ definition. The GENERATED_UCLASS_BODY() / GENERATED_USTRUCT_BODY() macros are required in reflected classes or structs, as they inject additional functions and typedefs into the class body.

The first property shown is ResourcesToGather, which is annotated with EditAnywhere and Category=Pawn. This means the property can be edited in any details panel in the editor, and will show up in the Pawn category. There are a couple of annotated functions marked with BlueprintCallable and a category, meaning they’ll be available to call from Blueprints.

As the MyTeamNum declaration shows, it’s fine to mix reflected and non-reflected properties in the same class, just be aware that the non-reflected properties are invisible to all of the systems that rely on reflection (e.g., storing a raw unreflected UObject pointer is usually dangerous since the garbage collector can’t see your reference).

Each of the specifier keywords (such as EditAnywhere or BlueprintCallable) is mirrored in ObjectBase.h with a short comment on the meaning or usage. If you’re not sure what a keyword does, Alt+G will usually work to take you to the definition in ObjectBase.h (they aren’t real C++ keywords, but Intellisense or VAX don’t seem to mind/understand the difference).

Check out the Gameplay Programming Reference for more information.

Limitations

UHT isn’t a real C++ parser. It understands a decent subset of the language and actively tries to skip any text that it can; only paying attention to reflected types, functions, and properties. However, some things can still confuse it, so you may have to reword something or wrap it in an #if CPP / #endif pair when adding a reflected type to an existing header. You should also avoid using #if/#ifdef (except for WITH_EDITOR and WITH_EDITORONLY_DATA) around any annotated properties or functions, since the generated code references them and will cause compile errors in any configuration where the define isn’t true.

Most common types work as expected, but the property system can’t represent all possible C++ types (notably only a few template types such as TArray and TSubclassOf are supported, and their template parameters cannot be nested types). UHT will give you a descriptive error message if you annotate a type that can’t be represented at runtime.

Using reflection data

Most game code can ignore the property system at runtime, enjoying the benefits of the systems that it powers, but you might find it useful when writing tool code or building gameplay systems.

The type hierarchy for the property system looks like this:

  • UField
    • UStruct
      • UClass (C++ class)
      • UScriptStruct (C++ struct)
      • UFunction (C++ function)
    • UEnum (C++ enumeration)
    • UProperty (C++ member variable or function parameter)
      • (Many subclasses for different types)

UStruct is the basic type of aggregate structures (anything that contains other members, such as a C++ class, struct, or function), and shouldn’t be confused with a C++ struct (that’s UScriptStruct). UClass can contain functions or properties as their children, while UFunction and UScriptStruct are limited to just properties.

You can get the UClass or UScriptStruct for a reflected C++ type by writing UTypeName::StaticClass() or FTypeName::StaticStruct(), and you can get the type for a UObject instance using Instance->GetClass() (it’s not possible to get the type of a struct instance since there is no common base class or required storage for structs).

To iterate over all members of a UStruct, use a TFieldIterator:

for (TFieldIterator<UProperty> PropIt(GetClass(), EFieldIteratorFlags::IncludeSuper); PropIt; ++PropIt)
{
	UProperty* Property = *PropIt;
	// Do something with the property
}

The template argument to TFieldIterator is used as a filter (so you can look at both properties and functions using UField, or just one or the other). The second argument to the iterator constructor indicates whether you only want fields introduced in the specified class/struct, or fields in the parent class/struct as well (the default); it doesn’t have any effect for functions.

Each type has a unique set of flags (EClassFlags + HasAnyClassFlags, etc…), as well as a generic metadata storage system inherited from UField. The keyword specifiers are usually either stored as flags or metadata, depending on whether they are needed in a runtime game, or only for editor functionality. This allows the editor-only metadata to be stripped out to save memory, while the runtime flags are always available.

You can do a lot of different things using the reflection data (enumerating properties, getting or setting values in a data-driven manner, invoking reflected functions, or even constructing new objects); rather than go in depth on any one case here, it’s probably easier to have a look thru UnrealType.h and Class.h, and track down an example of code that does something similar to what you want to accomplish.

A peek behind the curtain

You can safely skip this section if you just want to use the property system, but knowing how it works helps motivate some of the decisions and limitations in headers that contain reflected types.

Unreal Build Tool (UBT) and Unreal Header Tool (UHT) act in concert to generate the data that is needed to power runtime reflection. UBT has to scan headers to do its job, and it remembers any modules that contain a header with at least one reflected type. If any of those headers have changed since the last compile, UHT is invoked to harvest and update the reflection data. UHT parses the headers, builds up a set of reflection data, and then generates C++ code containing the reflection data (contributing to a per-module .generated.inl), as well as various helpers and thunk functions (per-header .generated.h).

One of the major benefits of storing the reflection data as generated C++ code is that it is guaranteed to be in sync with the binary. You can never load stale or out of date reflection data since it’s compiled in with the rest of the engine code, and it computes member offsets/etc… at startup using C++ expressions, rather than trying to reverse engineer the packing behavior of a particular platform/compiler/optimization combo. UHT is also built as a standalone program that doesn’t consume any generated headers, so it avoids the chicken-and-egg issues that were a common complaint with the script compiler in UE3.

The generated functions include things like StaticClass() / StaticStruct(), which make it easy to get reflection data for the type, as well as thunks used to call C++ functions from Blueprints or network replication. These must be declared as part of the class or struct, which explains why a GENERATED_UCLASS_BODY() or GENERATED_USTRUCT_BODY() macro is included in your reflected types, as well as the #include "TypeName.generated.h" which defines those macros.

Blueprint Editor Tips and Tricks

Originally posted on the Unreal Engine blog.

Context is King

When you’re working with the blueprint editor, it’s important to remember that context is king. If you right-click on a blueprint graph, you’ll get a list of actions that make sense in the context of the current blueprint and graph.

For example in an Actor based blueprint, you’ll get offered actions that target an Actor, as well as ones that work in any context. Similarly, if you drag off of a pin and let go on the graph background, the action menu will be filtered to actions that take or provide a value matching the pin type and direction.

The current context filter is always displayed at the top of the action menu, which can be helpful if you don’t see something you were expecting to find.

Blueprint Editor Action Menu
Blueprint Editor Action Menu

This filtering helps you find actions and place nodes quickly, but it requires a bit of forward planning that doesn’t always match the way you think when solving a problem. You need a specific object before you can see actions for that object (e.g., you need a static mesh component before you can look for SetStaticMesh).

The palette exists to help when you come up with the action before the target; it isn’t filtered by context and contains all possible actions.  You can drag-and-drop nodes from the palette into the graph, just like you can drag your custom functions from the My Blueprint list.  You can also disable the filtering in the action menu using the check box on the top right, but don’t forget about it, since it will stay unfiltered until you check the box again.

Searching

You can search in the current blueprint or in all blueprints in your project (even unloaded blueprints!) using Ctrl+F or Ctrl+Shift+F. Almost anything can be searched, including references to a variable or function, comments, literal values, etc…  There are also shortcuts to search for function and variable references in the context menu for your variables and user defined functions.

Cheat Sheet

The blueprint editor has a lot of productivity boosting shortcuts built-in, and while many will come naturally as you use the editor, others are a little bit hidden. These gems are well worth learning, so here is a cheat sheet of controls, keyboard shortcuts, and some useful actions:

Blueprint editor cheatsheet
Click to download: Color Version | Black/White Version

Fun with variables

You can create a new variable from any data pin by using the right click option ‘Promote to Variable’, which will automatically be hooked up to read/write it. As the cheat sheet mentions, there are also several useful actions when dragging variables from the My Blueprint tab:

  • Drag onto a pin: Places and connects a new get/set as appropriate
  • Drag to an existing variable get/set node: Changes it to read/write the dragged variable
  • Drag into the graph background: Offers a menu to get/set it; use Ctrl or Alt to force a Get or Set
  • Drag inside My Blueprint: Reorder the variables in a category or change categories

You can also clean up unused variables using ‘Edit..Delete Unused Variables’ to help tidy up your BP.

 

Unreal Engine 4 released!

I’ve been working on UE4 for the majority of my time at Epic, and I can finally talk publically about all the cool stuff that went into it! We announced (and launched) a subscription plan last week at GDC 2014.

You can get access to the UE4 editor, tools, and entire source code for a $19 per month subscription (and 5% gross royalty if you make money off of it). We’ve also got a bunch of code and content examples, with more coming over time. The EULA and subscription terms are concise and reasonable; you can cancel at any time and keep using what you’ve downloaded, renew when you want, etc…

On a related note: I will also be blogging at unrealengine.com, starting today with a post on Blueprint Editor Tips and Tricks, including a printable cheat sheet of BP editor controls / shortcuts (Cheat Sheet or Printable version).