The entropy of the universe is, essentially, traced by the number of photons in the universe since the production of photons goes hand in hand with processes that increase entropy. Now, so far as the expansion rate of the universe is concerned, the entropy of the universe is equal to the number of photons in the cosmic background radiation field. This has been constant since before the time the universe was 1 second old, and may have been 'fixed' by events that happened near the Inflation / GUT Phase Transition some 10^-34 seconds after the Big Bang. For the first 1/2 a million years, the expansion rate was determined by radiation pressure, and the scale increased at a rate of t^1/2. Once the universe became transparent to radiation in the cosmic background field, the expansion rate became dominated by the pressure of matter, and has been increasing at a rate of t^2/3 since about 1 million years after the Big Bang.
Since the entropy of the universe is still dominated by the cosmic background radiation, the process has remained essentially 'adiabatic' throughout the current age of the universe. However, as matter within stars is converted into light by fusion, additional entropy is added to the universe, so it is true that the universe is inexorably 'running down', and there is a definite arrow to the 'direction' in which entropy is slowly increasing for any given system in the universe. However, although the entropy of the universe as measured by its total background photon population is increasing, the photons contributed by stellar fusion is utterly negligable compared to the existing cosmic background photons which dominate the entropy of the universe. This is why the expansion can still be considered as a constant-entropy process, even though stars continue to increase the entropy slightly each year.
Also, in some sense the total entropy of the universe may be globally constant despite what the stars are doing. This is because the fusion process produces entropy at the expense of the gravitational field of each star which is doing work to drive the fusion process. The gravitational field of the universe may, in some sense, be considered as a source of 'negative entropy' so that the entropy produced by individual stars, is offset by borrowing from the entropy of the gravitational field of the universe so that the net change in entropy is actually zero, when averaged over the entire universe as a 'system'.
In this light, Time's Arrow defined by entropy increase, may have some kind of meaning for small parts of the universe at the scale of planets, stars and even galaxies, but may become ill-defined when averaged over the entire universe...at least this is how I understand the issue...and I confess, I am probably not stating the arguments correctly. There are many fine articles about Time's Arrow and the entropy of the universe in print by Paul Davies and even Stephen Hawking.