DVD Studio Pro lets you put scaled-down video in menus, but unfortunately the scaling quality is simply not up to professional standards (see above screenshots). But there is a workaround! The problem is that DVDSP uses the same low-quality scaling algorithm as Final Cut Pro/Express. But for more professional results, I developed a high quality scaling algorithm. All CGM plugins calculate the average of the pixels that have been removed while scaling down video. This is the important quality advantage. The solution for getting professional results is easy: use Adobe Photoshop for converting the DVDSP templates and adding the titles with their highlights. Then do the final compositing in Final Cut and export the result as an MPEG2 movie.

In DVD Studio Pro select the desired template by double-clicking. For this tutorial I have chosen the "Frames Cover" template. A new folder will be created in the Assets window called Templates. Open this folder, control-click (meaning hold down the control key while clicking) onto the background image and select Reveal In Finder. This reveals where the image files for this template are located on your hard disk, opening the folder on your desktop . Now you can load them into Photoshop and design your own menu template. I have added shadows to the horizontal lines and to the text.
When you are ready, flatten the background layers to one layer. Now there are only two layers left: the glass frame layer which has transparence areas (see upper picture) and the background layer (see lower picture). Save your template as a Photoshop file in your own documents folder on your hard disk (not into the DVDSP templates folder) and import it into Final Cut Pro/Express.
Open a new timeline for your motion menu. Place the background layer of the Photoshop file onto track 1 and the glass frame layer on track 3. The video could be placed on track 2. When you swich off the 3rd glass frame layer temporarily, the editing and previewing will be much faster. This way I produced a 30 second overview for my motion menu.
The next step is to scale this movie to 50% of its original size to fit inside the glass frame. Add the video filter CGM Basic 3D (FF) (located in: Video Filters / CGM DVE 3D) to your first video scene in track 2 and adjust the Scale size to the inner width of your glass frame (in the example 50%). Move the video to the upper left side into the glass frame by adjusting the sliders Horizonal and Vertical (in the example both are set to -12). If necessary, you can modify the sharpness of the scaled video by adjusting the slider Filter. The preset will be OK in most cases. Copy and paste the modified CGM Basic 3D to all videoclips in track 2.
How to add video transitions to limited areas: Add a CGM video transition (in the example CGM Oscillate) as a transition between two video scenes in track 2 and define the crop area in the Crop section by using the controls Center of Crop, Left/Right and Top/Bottom. Enable the function Stay Inside Crop Area. Nothing will be visible outside the crop area, better to say outside your glass frame video layer.
You'll get a more professional result by adding the Flicker Filter (located in: Video Filters / Video) to the glass frame and background layers. The best Amount is Flicker Filter (minimal). Export the whole composition as an MPEG2 movie and use this as background movie for your motion menu in DVDSP.