The mutuallyExclusive attribute cannot be enabled for an ace:buttonGroup with ace:checkboxButtons, if the checkboxButton components are inside an h:panelGrid. Removing the h:panelGrid fixes the issue.
This is not an issue if using ace:radioButtons inside the h:panelGrid.

        <ace:buttonGroup id="cbButtonGroup" header="Using ace:checkboxButton" mutuallyExclusive="true">
            <h:panelGrid id="panelGrid" columns="3" width="60%">
                <ace:checkboxButton value="#{buttonGroupBean.cb1}" label="#{buttonGroupBean.cbValueDescription1}">
                    <ace:ajax execute="@this" event="activate" render="@form"/>
                </ace:checkboxButton>
                <ace:checkboxButton value="#{buttonGroupBean.cb2}" label="#{buttonGroupBean.cbValueDescription2}">
                    <ace:ajax execute="@this" event="activate" render="@form"/>
                </ace:checkboxButton>
                <ace:checkboxButton value="#{buttonGroupBean.cb3}" label="#{buttonGroupBean.cbValueDescription3}">
                    <ace:ajax execute="@this" event="activate" render="@form"/>
                </ace:checkboxButton>
            </h:panelGrid>
        </ace:buttonGroup>

This can be reproduced with the ButtonGroup / ButtonGroup Only test app from: http://server.ice:8888/svn/repo/qa/trunk/Regression-Icefaces2/Sparkle/Manual/buttonGroup