RefPtr and PassRefPtr Basics

RefPtr and PassRefPtr Basics

Darin Adler

Version 4, 2010-08-27

History

Many objects in WebKit are reference counted. The pattern used is that classes have member functions ref and deref that increment and decrement the reference count. Each call to ref has to be matched by a call to deref. When the function is called on an object with a reference count of 1, the object is deleted. Many classes in WebKit implement this pattern by deriving from the RefCountedclass template.

Back in 2005, we discovered that there were many memory leaks, especially in HTML editing code, caused by misuse of ref and deref calls.

We wanted to use smart pointers to mitigate the problem. However, some early experiments showed that smart pointers led to additional manipulation of reference counts that hurt performance. For example, for a function that took a smart pointer as a parameter and returned that same smart pointer as a return value, just passing the parameter and returning the value would increment and then decrement the reference count two to four times as the object moved from one smart pointer to another. So we looked for an idiom that would let us use smart pointers and avoid this reference count churn.

The inspiration for a solution came from the C++ standard class template auto_ptr. These objects implement a model where assignment is transfer of ownership. When you assign from one auto_ptrto another, the donor becomes 0.

Maciej Stachowiak devised a pair of class templates, RefPtr and PassRefPtr, that implement this scheme for WebCore’s intrusive reference counting.

Raw pointers

When discussing smart pointers such as the RefPtr class template we use the term raw pointer to refer to the C++ language’s built in pointer type. Here’s the canonical setter function, written with raw pointers:

// example, not preferred style

class Document {
    ...
    Title* m_title;
}

Document::Document()
    : m_title(0)
{
}

Document::~Document()
{
    if (m_title)
        m_title->deref();
}

void Document::setTitle(Title* title)
{
    if (title)
        title->ref();
    if (m_title)
        m_title->deref();
    m_title = title;
}

RefPtr

RefPtr is a simple smart pointer class that calls ref on incoming values and deref on outgoing values.RefPtr works on any object with both a ref and a deref member function. Here’s the setter function example, written with RefPtr:

// example, not preferred style
 
class Document {
    ...
    RefPtr<Title> m_title;
}

void Document::setTitle(Title* title)
{
    m_title = title;
}

Use of RefPtr alone can lead to reference count churn.

// example, not preferred style; should use RefCounted and adoptRef (see below)
 
RefPtr<Node> createSpecialNode()
{
    RefPtr<Node> a = new Node;
    a->setSpecial(true);
    return a;
}

RefPtr<Node> b = createSpecialNode();

For purposes of this discussion, lets assume that the node object starts with a reference count of 0 (more on this later). When it’s assigned to a, the reference count is incremented to 1. The reference count is incremented to 2 to create the return value, then decremented back to 1 when a is destroyed. Then the reference count is incremented to 2 to create b, and then decremented back to 1 when the return value of createSpecialNode is destroyed.

(If the compiler implements the return value optimization, there may be one less increment and decrement of the reference count.)

The overhead of reference count churn is even greater when both function arguments and return values are involved. The solution is PassRefPtr.

PassRefPtr

PassRefPtr is like RefPtr with a difference. When you copy a PassRefPtr or assign the value of aPassRefPtr to a RefPtr or another PassRefPtr, the original pointer value is set to 0; the operation is done without any change to the reference count. Let’s take a look at a new version of our example:

// example, not preferred style; should use RefCounted and adoptRef (see below)

PassRefPtr<Node> createSpecialNode()
{
    PassRefPtr<Node> a = new Node;
    a->setSpecial(true);
    return a;
}

RefPtr<Node> b = createSpecialNode();

The node object starts with a reference count of 0. When it’s assigned to a, the reference count is incremented to 1. Then a gets set to 0 when the return value PassRefPtr is created. Then the return value is set to 0 when b is created.

However, as the Safari team learned when we started programming with PassRefPtr, the rule that a pointer becomes 0 when it’s assigned to another variable can easily lead to mistakes.

// warning, will dereference a null pointer and will not work
 
static RefPtr<Ring> g_oneRingToRuleThemAll;

void finish(PassRefPtr<Ring> ring)
{
    g_oneRingToRuleThemAll = ring;
    ...
    ring->wear();
}

By the time wear is called, ring is already 0. To avoid this, we recommend PassRefPtr only for function argument and result types, copying arguments into RefPtr local variables.

static RefPtr<Ring> g_oneRingToRuleThemAll;

void finish(PassRefPtr<Ring> prpRing)
{
    RefPtr<Ring> ring = prpRing;
    g_oneRingToRuleThemAll = ring;
    ...
    ring->wear();
}

Mixing RefPtr and PassRefPtr

Since we recommend use of RefPtr in all cases except when passing arguments to or returning values from a function, there will be times when you have a RefPtr and wish to transfer ownership asPassRefPtr does. RefPtr has a member function named release that does the trick. It sets the value of the original RefPtr to 0 and constructs a PassRefPtr, without changing reference counts.

// example, not preferred style; should use RefCounted and adoptRef (see below)
 
PassRefPtr<Node> createSpecialNode()
{
    RefPtr<Node> a = new Node;
    a->setCreated(true);
    return a.release();
}

RefPtr<Node> b = createSpecialNode();

This keeps the efficiency of PassRefPtr while reducing the chance that its relatively tricky semantics will cause problems.

Mixing with raw pointers

When using a RefPtr to call a function that takes a raw pointer, use the get function.

printNode(stderr, a.get());

However, many operations can be done on a RefPtr or PassRefPtr directly, without resorting to an explicit get call.

RefPtr<Node> a = createSpecialNode();
Node* b = getOrdinaryNode();

// the * operator
*a = value;

// the -> operator
a->clear();

// null check in an if statement
if (a)
    log("not empty");

// the ! operator
if (!a)
    log("empty");

// the == and != operators, mixing with raw pointers
if (a == b)
    log("equal");
if (a != b)
    log("not equal");

// some type casts
RefPtr<DerivedNode> d = static_pointer_cast<DerivedNode>(a);

Normally, RefPtr and PassRefPtr enforce a simple rule; they always balance ref and deref calls, guaranteeing a programmer can’t miss a deref. But in the case where we have a raw pointer, already have a reference count, and want to transfer ownership the adoptRef function should be used.

// warning, requires a pointer that already has a ref
RefPtr<Node> node = adoptRef(rawNodePointer);

To transfer from a RefPtr to a raw pointer without changing the reference count, PassRefPtr provides the leakRef function.

// warning, results in a pointer that must get an explicit deref
RefPtr<Node> node = createSpecialNode();
Node* rawNodePointer = node.release().leakRef();

Since leakRef is rarely used, it’s provided only in the PassRefPtr class, hence the need to call release, then leakRef.

RefPtr and new objects

In the examples in this discussion, we talked about objects with a reference count of 0. However, for efficiency and simplicity, the RefCounted class doesn't use a reference count of 0 at all. Objects are created with a reference count of 1. The best programming idiom to use is to put such objects right into a RefPtr to make it impossible to forget to deref the object when done with it. This means that anyone calling new on such an object should immediately call adoptRef. In WebCore we use functions named create instead of direct calls to new.

// preferred style
 
PassRefPtr<Node> Node::create()
{
    return adoptRef(new Node);
}

RefPtr<Node> e = Node::create();

Because of the way adoptRef and PassRefPtr are implemented, this is an efficient idiom. The object starts with a reference count of 1 and no manipulation of the reference count happens at all.

// preferred style
 
PassRefPtr<Node> createSpecialNode()
{
    RefPtr<Node> a = Node::create();
    a->setCreated(true);
    return a.release();
}

RefPtr<Node> b = createSpecialNode();

The node object is put into a PassRefPtr by a call to adoptRef inside Node::create, then passes into aand is released and passes into b, all without touching the reference count.

The RefCounted class implements a runtime check so we get an assertion failure if we create an object and call ref or deref without first calling adoptRef.

Guidelines

We’ve developed these guidelines for use of RefPtr and PassRefPtr in WebKit code.

Local variables

• If ownership and lifetime are guaranteed, a local variable can be a raw pointer.
• If the code needs to hold ownership or guarantee lifetime, a local variable should be a RefPtr.
• Local variables should never be PassRefPtr.

Data members

• If ownership and lifetime are guaranteed, a data member can be a raw pointer.
• If the class needs to hold ownership or guarantee lifetime, the data member should be aRefPtr.
• Data members should never be PassRefPtr.

Function arguments

• If a function does not take ownership of an object, the argument should be a raw pointer.
• If a function does take ownership of an object, the argument should be a PassRefPtr. This includes most setter functions. Unless the use of the argument is very simple, the argument should be transferred to a RefPtr at the start of the function; the argument can be named with a “prp” prefix in such cases.

Function results

• If a function’s result is an object, but ownership is not being transferred, the result should be a raw pointer. This includes most getter functions.
• If a function’s result is a new object or ownership is being transferred for any other reason, the result should be a PassRefPtr. Since local variables are typically RefPtr, it’s common to callrelease in the return statement to transfer the RefPtr to the PassRefPtr.

New objects

• New objects should be put into a RefPtr as soon as possible after creation to allow the smart pointers to do all reference counting automatically.
• For RefCounted objects, the above should be done with the adoptRef function.
• Best idiom is to use a private constructor and a public create function that returns aPassRefPtr.

Improving this document

We should add answers to any frequently asked questions are not covered by this document. One or more of the following topics could also be covered by this document.

• The “protector” idiom, where a local RefPtr variable is used to keep an object alive.
• Perils of programming with TreeShared.
• Our desire to eliminate TreeShared and instead have m_firstChild and m_next be ListRefPtr or the equivalent.
• How we we mix reference counting with garbage collection to implement the DOM and the JavaScript and Objective-C DOM bindings.
• Comparison of our intrusive reference counting with other schemes such as the external reference counting in Boost shared_ptr.
• The OwnPtr class template, and how it can be used with PassOwnPtr and adoptPtr.
• The OwnArrayPtr class template, and PassOwnArrayPtr.
• The RetainPtr class template, and the lack of a PassRetainPtr.
• The ListRefPtr class template.

If you have any comments on the above or other ideas about improving the clarity, scope, or presentation, please send mail to the WebKit mailing list.