Perforce Notes

Russ Bateman
21 March 2011
last update:

Table of Contents

Install Perforce
Changing source branches
Synchronizing (updating)
Forcing synchronization
Changing and committing files
Adding new files
Deleting files
Reverting changes
File status hints
Who's editing a file?
Juggling multiple Perforce clients
Appendix: Some additional notes
Appendix: Perforce-Eclipse integration
Appendix: A sample exercise
Appendix: p4v: The GUI client
Appendix: Perforce and Eclipse

Install Perforce

This is a commercial, proprietary source code management system. You can use a visual GUI or command-line client to manage source code check-outs, updates and commits.

  1. Set up Perforce by going to their site http://www.perforce.com/ and clicking the Free Download link. Click Download Perforce and then chose Linux.
  2.  
  3. Choose the command-line client (p4 not p4v). Choose your host's word width, then click the last Download button. What comes down is a tiny binary, p4.
  4.  
  5. Save the file at /home/user/bin, /usr/local/bin or wherever you like best.

Changing source branches

To change what you've got from Perforce, simply modify the client specification via

p4 client

To switch branches, for instance, you have only to...

  1. Run p4 client
  2. Change the lone View: line in the client specification referring to the branch, e.g.: from "2011.6" to "Integration".
  3. Exit p4 client with update.
  4. Run p4 sync and, possibly p4 sync -f ... in places where you don't get exactly what you should. See "Forcing synchroniztion".

Templates

If you're excluding files like .png, .gif and .jpg from templates then you're removing most of the grief caused by the templates. The best solution is to prepare a shared directory for accessing templates, per office. At present, there is no working implementation of this.

Synchronization (updating)

This is a simple matter; you can update everything you've told Perforce is a concern to you (via p4 client) by just

p4 sync

If you're in a specific subdirectory, do this:

p4 sync ...

Forcing synchronization

When encountering build errors because of missing directories or files in the Snapfish sources, as long as local.properties is accurate, you may find the need to force synchronization.

Every time you perform synchronization, Perforce keeps track of what files (and versions of files) it thinks you have in your hos filesystem. The next time you synchronize, it compares what it thinks you have with what it has that's new and only sends down the items for which there is a delta. It does not examine your local filesytem to ascertain the truth.

This is calculated to lessen traffic on the wire and pressure on Perforce itself. You should not over-use this option with Perforce.

For this reason, some things will get out of sync, if rarely. If you're changing product branches frequently, this is likely to happen less rarely and you'll have to resort to forcing things.

The use of the -f option says, in effect, "Disregard what you think I have and give me everything my client specification says to give me."

To force resynchronization of everything, ...

p4 sync -f

If you're reacting to a build error and you understand where the disconnect is occurring, force resynchronization only of that portion that's necessary by going to the top of the filesystem you wish to resynchronize. This does just your current working directory and everything under it (assuming all these are under Perforce control).

p4 sync -f ...

Don't do this!!!

Okay, now that you've just read how to force synchronization, try never to use it. Daniel says he never has to (but then, he's just the little wizard, isn't he?). Here are some work-arounds to try first:

p4 diff -se # show unopened files that differ from the depot revision p4 diff -se | p4 -x- edit # fix files modified without being formally opened for edit p4 diff -sd # show missing files p4 diff -sd | p4 -x- sync -f # force-sync missing files

Changing and committing files

Assuming you have synchronized files from a Perforce repository, you must tell Perforce that you wish to edit the file in order, later, to commit changes.

p4 edit filepath //depot/somewhere/snapfish-core/blah/filepath - opened for edit

To commit a file you've edited, ...

p4 submit filepath

This will launch vi just as other source code-control software permitting you to explain the reason for committing the change.

Rules for committing

These depend on the repository owner, i.e.: your company. In the illustrations below you'll see my company's policy in the form of VID, PID, FF, etc.

Organizational policy can disallow a submission. This cannot be described here.

Adding new files

To add a file you've created, use

p4 add filepath //depot/somewhere/snapfish-core/blah/filepath - opened for add

As soon as you like, you may commit it using p4 submit.

Deleting files

To delete a file currently under Perforce control, do

p4 delete //depot/<path>/foo.txt

Reverting changes

When you "check out" a file for editing, Perforce creates a changelist that only it knows about. This is use to later commit changes (see Committing files above).

If you opened a file for editing and no longer wish to make changes, whether or not you modified the file, use p4 revert <filename> to "drop your check-out".

p4 revert only backs out changes from an unsubmitted (uncommitted) changelist. If you kept track of Perforce's echoed change number, that number can be very useful in backing out changes. If the change you want to back out was very recent, you can also get a list of such changes (-m 5 limits the list to 5 here):

p4 changes -m 5 -u jay.johnson Change 414336 on 2011/04/08 by [email protected] ' VID: PID: BID: FF: Added this to xxxx.java file...' Change 414334 on 2011/04/08 by [email protected] ' VID: PID: BID: FF: Added a' Change 414333 on 2011/04/08 by [email protected] ' VID: PID: BID: FF: Added a ' Change 414330 on 2011/04/08 by [email protected] ' VID: PID: BID: FF: '

Begin reverting...

To begin the process, you must have the change number. Subtract 1 from it to produce the number of the change just prior to the one you wish unapply. Synchronize your subdirectory to that change. Understand that if, for a given change, you checked in more than one file, every file (or subdirectory) that was part of that change will be back-rev'd.

After back-rev'ing, "open" the file(s) for editing. Without making changes to the file(s), re-synchronize, then resolve, then re-submit. This sequence accomplishes erasing the change you made. Here are the steps illustrated. Notice that 414335 is 1 less than 414336 above.

p4 sync @414335 p4 edit xxxx.java [yyyy.java etc.] p4 sync p4 resolve -ay p4 submit

Trick to back out older change

There is even a trick to backing out a change made subsequent to which many other changes were made. The trick is to re-synchronize to the change number after backing out that change. Let's assume that the change number to be erased is 1000. The major differences between this operation and the simpler one already described are in bold below:

p4 sync @999 p4 edit xxxx.java [yyyy.java etc.] p4 sync @1000 p4 resolve -ay p4 sync p4 resolve p4 submit

(Yes, it's unclear why one synchronizes to 1000 instead of 1001, but that's what the example says. I'll update once I have to do this case myself.)

p4 status

There is no p4 status command, but this works for finding untracked files from the current working directory down:

find -type f ! -name '*~' -print0| xargs -0 p4 fstat 2>&1|awk '/no such file/{print $1}'

File status hints

There is no p4 status command as in cvs, svn and git. However, there are some commands that will show somewhat similar things about your filesystem copy to put you at ease. The responses shown demonstrates there is nothing to worry about.

p4 opened (Show all files opened for editing) p4 diff -sa (Show opened files that are different from the depot, or missing) File(s) not opened on this client. p4 diff -sb (Show files opened for integration that have been resolved, but later modified) File(s) not opened on this client. p4 resolve -n (Show files in need of resolving) No file(s) to resolve.

To find files you've failed to add (p4 add), use this. What comes out is a list of files that aren't tracked by Perforce—there will be many and many that should not.

find . -type f -print0 | xargs -0 p4 fstat > dev/null ./rsync/rsyncd.global.conf.bak - file(s) not in client view.

Who's editing a file?

When you open a file for editing using p4 edit, Perforce tells you who else has that file open via the format <user>@<workspace>. However, you likely will not know who this really is. The following command will dump the first few lines of his Perforce client configuration:

p4 user -o <user>

...where <user> is the first string before '@' in what Perforce echoed back.

Juggling multiple Perforce clients

You can have as many Perforce clients as you like. Normally your p4 client comes from the P4CLIENT environment variable. Those using multiple clients do the following.

Set P4CONFIG in your .bashrc to something like .p4config.

If you set this environment variable then instead of consulting environment for the Perforce client, user, password and port, Perforce will search your current working directory (and every directory above this) for a file named the same as the value of ${P4CONFIG}. If it finds such a file, it'll read that file for its configuration.

By way of example, to maintain two different clients you create two clients (make sure their root directories are different), then put a .p4config in the root of the client with the content

P4CLIENT=client-name

Create a separate client for every branch you wish to maintain or work with. When you're done with a branch, delete your client. Keep a separate client for shared resources, like modules, templates, dna_binaries, etc.

Daniel Rogers' .p4config files look like:

P4CLIENT=dsrogers-creamy

or

P4CLIENT=dsrogers-environment

Appendix: Some additional notes

Here are some brief notes on additional Perforce topics that might later be integrated into the article above. Most of these are from a session with Ravi (Vanga Palli Ravinder Kumar).

Merging

Steps for merging. These are from the Snapfish wiki. This is interactive as you must pick your way through the merge.

  1. p4 integrate
  2. p4 resolve
  3. p4 submit

List files in filesystem under Perforce control

Do this to see if a file or files (or all files—no argument) is (are) under Perforce control and what version you have:

p4 have filename]

List files open for edit

p4 opened lists the file on which p4 edit has been done.

Saving database space on the server

Removing lines from client spec (p4 client) will result in erasing these from the database. It takes time to do this on the development host—why so many developers don't take time to do it. Ravi has a script for managing branches that solves this problem without spending time.

List Perforce users

p4 users. To see details about self, do p4 users | russ.

Examine what has happened to a file

To see what's been happening to a file use the following command. Adding option -s shortens what's listed dramatically.

p4 filelog [-s] filename

Create a new changelist

p4 change creates a new one; be sure to record the number for later reference.

diff of a file's versions

Display the differences between two versions of a file. Yields the expected, Unix-style format.

$ p4 diff file.ext{#24,#25}

Appendix: Perforce-Eclipse integration

I don't advise you to integrate Perforce into Eclipse unless you're willing to endure considerable slowness on start-up—about 30 seconds in my case. (What's slow is the required Eclipse Mylyn software.) You don't really need it and can do everything from the command line.

To uninstall Perforce-Eclipse integration

Use Help -> About Eclipse -> Installation Details. Select all marked "Perforce" and then click the Uninstall button.


Download the Perforce-Eclipse integration plug-in the usual way: Help -> Install New Software... -> Add and name it "Perforce plug-in" with this URL, which supports Helios (3.6):

    http://www.perforce.com/downloads/http/p4-eclipse/install/3.6

Click the associated bits of functionality as desired. Mylyn comes free with Helios and you may find yourself using it down the road. It's a task manager and is required for integration with Rally as well. (So, if it's there, if Rally is using it, maybe the Perforce integration will be useful too. It will not take up memory or anything but diskspace as long as you don't actually use it.)

Importing a project into Eclipse

This isn't greatly difficult. Mostly, be careful not to check the box to copy the sources into the workspace. (I haven't modified code yet, so I'm unsure of all the "read-only" versus "not read-only" states here.)

There are two ways to get projects into Eclipse using Perforce integration. In this case, the projects imported are not read-only, i.e.: the sources are ready for modification.

  1. Open the the Perforce perspective and, in the P4 Connections view, go to the project and select either Get Latest Revision or Get Revision. In this case, the project is imported to your Perforce workspace (not to be confused with any Eclipse workspace).
  2.  
  3. In the P4 Connections view, select the project and choose Import as Project or Import as Project with Name....

    In this case the project will go to the Perforce workspace and will also be visible in the Eclipse Package Explorer view. The project gets imported into the Eclipse workspace associated with IDE (but it won't get copied there; the project will still reside in the Perforce workspace).

Appendix: A sample exercise...

This is an illustration of creating a new file and editing three others in an active code base.

[email protected]:~/snapcat/> p4 add 2011/8/javasrc/com/snapfish/be/rest/server/handlers/CLookupByEmailHandler.java //depot/Snapfish/2011/8/javasrc/com/snapfish/be/rest/server/handlers/CLookupByEmailHandler.java#1 - opened for add [email protected]:~/snapcat> ll 2011/8/components/snapfish-beapi/be_api*.xsd 2011/8/javasrc/resources/beapiconfig.properties.in -rw------- 1 russ russ 172717 2011-04-20 15:13 2011/8/components/snapfish-beapi/be_api_types.xsd -rw------- 1 russ russ 43220 2011-04-20 15:13 2011/8/components/snapfish-beapi/be_api.xsd -rw------- 1 russ russ 14319 2011-04-18 15:27 2011/8/javasrc/resources/beapiconfig.properties.in [email protected]:~/snapcat> p4 edit 2011/8/components/snapfish-beapi/be_api*.xsd 2011/8/javasrc/resources/beapiconfig.properties.in //depot/Snapfish/2011/8/components/snapfish-beapi/be_api_types.xsd#8 - opened for edit ... //depot/Snapfish/2011/8/components/snapfish-beapi/be_api_types.xsd - must sync/resolve #9 before submitting //depot/Snapfish/2011/8/components/snapfish-beapi/be_api.xsd#5 - opened for edit ... //depot/Snapfish/2011/8/components/snapfish-beapi/be_api.xsd - must sync/resolve #6,#7 before submitting ... //depot/Snapfish/2011/8/components/snapfish-beapi/be_api.xsd - also opened by [email protected]_ubuntu //depot/Snapfish/2011/8/javasrc/resources/beapiconfig.properties.in#5 - opened for edit ... //depot/Snapfish/2011/8/javasrc/resources/beapiconfig.properties.in - must sync/resolve #6 before submitting

I must use p4 resolve to combine the contents of the three files I marked for edit because if, for example, Kate Middleton marked be_api.xsd for edit, then added something to it followed by me doing the same thing, committing my copy would wipe her changes out.

What if my copy dates back two weeks and there are yet other changes before Kate's that weren't in mine since mine is so old? Can I simply do p4 resolve be_api.csd and Perforce will not see in my files what's old (and now changed) besides my additions as changes I want to make?

Appendix: p4v: The GUI client

Here are random notes on this fairy-boy tool that I don't use much, but it's really effective at doing visual merges.

See P4V File Icons for the details as to what the icons in this tool mean: it's very important to understand them as they tell you immediately what state a file's in (you've modified it, someone else has already committed modifications since you opened it for edit, etc.).

Steps for running the visual merge tool

  1. Look in the right-side pane for the Pending tab.
  2. Expand default.
  3. Right-click any file that's in need of update to get changes another user may have put in since you opened this file for edit.
  4. Right-click the file and choose Get Latest Revision.
  5. If something's red (probably a red question mark decoration of the icon), you have to resolve the file. Choose to Resolve it.
  6. Click Resolve.
  7. Click Run merge tool.
  8.  
    - The left pane is the file as it was submitted by another (i.e.: whose changes beat yours into Perforce),
     
    - the middle pane is the base neutral between the two versions, and
     
    - the right pane is the file with your changes.
     
    What you need to do is merge the changes from the left pane into your new version, the resolved file, in the bottom pane!
     
    You'll notice that the Perforce visual mergtool already merged what it identified as "no-brainer" integrations, moving your new code into the result file in the bottom window. You can configure it not to do this, but why? All these assumptions are marked below anyway as you see in this example:

     
  9. Step through the changes (moves focus in all three panes) by clicking the large, aquamarine-colored right arrow button in the toolbar above. These changes will be in your version if you approve what Perforce has done.
  10. When happy, click the Save (blue diskette) icon in the toolbar above.
  11. Dismiss this mergetool window and confirm that you want to keep the changes.

Appendix: Perforce and Eclipse

These notes were offered in response to a question in a forum.

Consider the following:

  1. Eclipse workspaces aren't portable or sharable, only projects are. Don't commit (p4 add/submit), for example, the workspace subdirectory or its .meta subdirectory thinking there's anything to be gained by it. Also in the workspace will be a Server subdirectory if you're doing web development. Don't try to store that in Perforce.
  2. Do commit .project, .classpath (both files) and .settings (a subdirectory).
  3. Take care how you structure the addition of third-party libraries. Make sure you add them with Java Build Path as an in-project navigation. Don't go outside the project for them or for source or Javadoc you add to them. This ensures that .classpath will not contain any host-specific paths.

    If you're cloudy on this point, the following comments, though made for Git, are relevant:

    http://www.javahotchocolate.com/tutorials/git-eclipse.html

  4. A colleague of mine always manages two separate Perforce client specifications so that he can do a p4 sync separately to try out what he's done in order to know when he's broken it. I personally have never broken the build in two years running, so I don't know what there is to be gained from this, but he swears by the practice.
  5. If the project is submitted completely and accurately to Perforce, then a new engineer should be able to
    1. p4 client
    2. p4 sync
    3. Launch Eclipse with a suitable workspace.
    4. Import the project into his workspace; to do this:
      1. File -> Import -> General -> Existing Projects into Workspace
      2. Next
      3. Navigate to the parent of the subdirectory that is the root of the project in your filesystem (filesystem under Perforce control)
      4. Select that project and click OK.
      5. Don't opt to bring that project's files and directories into the Eclipse workspace; leave it where it is, where Perforce will be managing it and where you'll issue Perforce commands (or just use p4v).
    5. If doing web development, create a Server.

    When I say command line, note that you can also use p4v if you like. I rarely do, but sometimes I do when I want to explore the rather humongous amount of code there is—only a tiny part of which I actually have to work on.