Japanese garden plants are gleaned from a palette of classic shrubs, trees and ground cover plants - the most evocative of these are junipers and pines trimmed in the cloud pruning style, bamboo in groves and the whole area carpeted in lovely emerald green moss.
The most well known types of plants for your Japanese style garden are small and well behaved - most gardens in Japan are tiny gemlike areas with only a few special perfectly placed plants.
Acer palmatum, the Japanese maple, is one of the best of all small trees for this type of gardening; slow growing, amenable to pruning into fantastical shapes, and long lived. The four seasons of its beauty are always awe inspiring.
Bamboo is a commonly grown plant in Japanese gardens. It's graceful, moving with the wind, and also shows the changes with the seasons, in winter the long bare stems show to good effect against a snowy background, and in summer the slender leaves rustle to make white noise that hides traffic sounds.
Grasses are a valuable plant for the same reasons, although smaller.
Junipers and other evergreens are sometimes pruned into cloud shapes; some talent is necessary to start the shaping process, and dedication is required for frequent pinching to keep the shape perfect.
Moss is one of my favorite types of plants - all it needs is some shallow soil to grow in, and a fairly reliably source of moisture. In spite of these criteria, it also has no objection to growing on straight stone, and can withstand considerable drought. After the dry spell ends and the rains return, the dried out moss plumps up and turns green again, like magic.