Linked files are assigned to the active workset, when worksets are in use naturally. When you select the link you can see the Workset parameter in the Properties Palette. Less obvious is that they also have a Type Property called, you guessed it, Workset. This can trip you up when you are trying alter visibilty graphics because they need to match.
The reason they have both is that the Type Parameter is for the definition of the link in the database and the instance parameter is for the actual link. There is a row in the Visibility Graphics dialog for each link as well as a single row beneath it for each instance of the link.
I say each instance because a link can be copied. If you import a file it is both the definition and the instance. If you copy it and put the second instance elsewhere you'll see that another row will appear beneath the first one and both will be under the primary entry. This is done to make it possible to alter each instance of the same linked file differently via Visibilty Graphics.
Experiencing control issues with a link? Remember to check the workset assignments.