Cut the potatoes into ¾-inch cubes. Bring a large pot of salted water to a boil. Add the potatoes and cook until tender, about 20 minutes. Drain and let cool slightly.
Meanwhile, in a large bowl, whisk the olive oil with the vinegar and mustard, then season with salt and pepper. Gently blend in the warm potatoes and the celery. Let cool to room temperature. Fold in the scallions and thyme, and adjust seasonings to taste. Serve at room temperature.
Notes
If you don't have white balsamic vinegar you can use white wine vinegar, which is a bit sharper. Reduce the amount slightly if you want a softer tasting dressing.
Adding the still-warm potatoes to the vinaigrette allows them to better absorb the flavors of the dressing.