Merge remote-tracking branch 'remotes/palmer/tags/palmer-for-master-4.2-sf1' into staging
Update my MAINTAINERS file entry
This contains a single patch to change my email address.
# gpg: Signature made Fri 01 Nov 2019 16:14:45 GMT
# gpg: using RSA key 00CE76D1834960DFCE886DF8EF4CA1502CCBAB41
# gpg: issuer "palmer@dabbelt.com"
# gpg: Good signature from "Palmer Dabbelt <palmer@dabbelt.com>" [unknown]
# gpg: aka "Palmer Dabbelt <palmer@sifive.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 00CE 76D1 8349 60DF CE88 6DF8 EF4C A150 2CCB AB41
* remotes/palmer/tags/palmer-for-master-4.2-sf1:
MAINTAINERS: Change to my personal email address
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
diff --git a/docs/arm-cpu-features.rst b/docs/arm-cpu-features.rst
new file mode 100644
index 0000000..1b367e2
--- /dev/null
+++ b/docs/arm-cpu-features.rst
@@ -0,0 +1,317 @@
+================
+ARM CPU Features
+================
+
+Examples of probing and using ARM CPU features
+
+Introduction
+============
+
+CPU features are optional features that a CPU of supporting type may
+choose to implement or not. In QEMU, optional CPU features have
+corresponding boolean CPU proprieties that, when enabled, indicate
+that the feature is implemented, and, conversely, when disabled,
+indicate that it is not implemented. An example of an ARM CPU feature
+is the Performance Monitoring Unit (PMU). CPU types such as the
+Cortex-A15 and the Cortex-A57, which respectively implement ARM
+architecture reference manuals ARMv7-A and ARMv8-A, may both optionally
+implement PMUs. For example, if a user wants to use a Cortex-A15 without
+a PMU, then the `-cpu` parameter should contain `pmu=off` on the QEMU
+command line, i.e. `-cpu cortex-a15,pmu=off`.
+
+As not all CPU types support all optional CPU features, then whether or
+not a CPU property exists depends on the CPU type. For example, CPUs
+that implement the ARMv8-A architecture reference manual may optionally
+support the AArch32 CPU feature, which may be enabled by disabling the
+`aarch64` CPU property. A CPU type such as the Cortex-A15, which does
+not implement ARMv8-A, will not have the `aarch64` CPU property.
+
+QEMU's support may be limited for some CPU features, only partially
+supporting the feature or only supporting the feature under certain
+configurations. For example, the `aarch64` CPU feature, which, when
+disabled, enables the optional AArch32 CPU feature, is only supported
+when using the KVM accelerator and when running on a host CPU type that
+supports the feature.
+
+CPU Feature Probing
+===================
+
+Determining which CPU features are available and functional for a given
+CPU type is possible with the `query-cpu-model-expansion` QMP command.
+Below are some examples where `scripts/qmp/qmp-shell` (see the top comment
+block in the script for usage) is used to issue the QMP commands.
+
+(1) Determine which CPU features are available for the `max` CPU type
+ (Note, we started QEMU with qemu-system-aarch64, so `max` is
+ implementing the ARMv8-A reference manual in this case)::
+
+ (QEMU) query-cpu-model-expansion type=full model={"name":"max"}
+ { "return": {
+ "model": { "name": "max", "props": {
+ "sve1664": true, "pmu": true, "sve1792": true, "sve1920": true,
+ "sve128": true, "aarch64": true, "sve1024": true, "sve": true,
+ "sve640": true, "sve768": true, "sve1408": true, "sve256": true,
+ "sve1152": true, "sve512": true, "sve384": true, "sve1536": true,
+ "sve896": true, "sve1280": true, "sve2048": true
+ }}}}
+
+We see that the `max` CPU type has the `pmu`, `aarch64`, `sve`, and many
+`sve<N>` CPU features. We also see that all the CPU features are
+enabled, as they are all `true`. (The `sve<N>` CPU features are all
+optional SVE vector lengths (see "SVE CPU Properties"). While with TCG
+all SVE vector lengths can be supported, when KVM is in use it's more
+likely that only a few lengths will be supported, if SVE is supported at
+all.)
+
+(2) Let's try to disable the PMU::
+
+ (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"pmu":false}}
+ { "return": {
+ "model": { "name": "max", "props": {
+ "sve1664": true, "pmu": false, "sve1792": true, "sve1920": true,
+ "sve128": true, "aarch64": true, "sve1024": true, "sve": true,
+ "sve640": true, "sve768": true, "sve1408": true, "sve256": true,
+ "sve1152": true, "sve512": true, "sve384": true, "sve1536": true,
+ "sve896": true, "sve1280": true, "sve2048": true
+ }}}}
+
+We see it worked, as `pmu` is now `false`.
+
+(3) Let's try to disable `aarch64`, which enables the AArch32 CPU feature::
+
+ (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"aarch64":false}}
+ {"error": {
+ "class": "GenericError", "desc":
+ "'aarch64' feature cannot be disabled unless KVM is enabled and 32-bit EL1 is supported"
+ }}
+
+It looks like this feature is limited to a configuration we do not
+currently have.
+
+(4) Let's disable `sve` and see what happens to all the optional SVE
+ vector lengths::
+
+ (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"sve":false}}
+ { "return": {
+ "model": { "name": "max", "props": {
+ "sve1664": false, "pmu": true, "sve1792": false, "sve1920": false,
+ "sve128": false, "aarch64": true, "sve1024": false, "sve": false,
+ "sve640": false, "sve768": false, "sve1408": false, "sve256": false,
+ "sve1152": false, "sve512": false, "sve384": false, "sve1536": false,
+ "sve896": false, "sve1280": false, "sve2048": false
+ }}}}
+
+As expected they are now all `false`.
+
+(5) Let's try probing CPU features for the Cortex-A15 CPU type::
+
+ (QEMU) query-cpu-model-expansion type=full model={"name":"cortex-a15"}
+ {"return": {"model": {"name": "cortex-a15", "props": {"pmu": true}}}}
+
+Only the `pmu` CPU feature is available.
+
+A note about CPU feature dependencies
+-------------------------------------
+
+It's possible for features to have dependencies on other features. I.e.
+it may be possible to change one feature at a time without error, but
+when attempting to change all features at once an error could occur
+depending on the order they are processed. It's also possible changing
+all at once doesn't generate an error, because a feature's dependencies
+are satisfied with other features, but the same feature cannot be changed
+independently without error. For these reasons callers should always
+attempt to make their desired changes all at once in order to ensure the
+collection is valid.
+
+A note about CPU models and KVM
+-------------------------------
+
+Named CPU models generally do not work with KVM. There are a few cases
+that do work, e.g. using the named CPU model `cortex-a57` with KVM on a
+seattle host, but mostly if KVM is enabled the `host` CPU type must be
+used. This means the guest is provided all the same CPU features as the
+host CPU type has. And, for this reason, the `host` CPU type should
+enable all CPU features that the host has by default. Indeed it's even
+a bit strange to allow disabling CPU features that the host has when using
+the `host` CPU type, but in the absence of CPU models it's the best we can
+do if we want to launch guests without all the host's CPU features enabled.
+
+Enabling KVM also affects the `query-cpu-model-expansion` QMP command. The
+affect is not only limited to specific features, as pointed out in example
+(3) of "CPU Feature Probing", but also to which CPU types may be expanded.
+When KVM is enabled, only the `max`, `host`, and current CPU type may be
+expanded. This restriction is necessary as it's not possible to know all
+CPU types that may work with KVM, but it does impose a small risk of users
+experiencing unexpected errors. For example on a seattle, as mentioned
+above, the `cortex-a57` CPU type is also valid when KVM is enabled.
+Therefore a user could use the `host` CPU type for the current type, but
+then attempt to query `cortex-a57`, however that query will fail with our
+restrictions. This shouldn't be an issue though as management layers and
+users have been preferring the `host` CPU type for use with KVM for quite
+some time. Additionally, if the KVM-enabled QEMU instance running on a
+seattle host is using the `cortex-a57` CPU type, then querying `cortex-a57`
+will work.
+
+Using CPU Features
+==================
+
+After determining which CPU features are available and supported for a
+given CPU type, then they may be selectively enabled or disabled on the
+QEMU command line with that CPU type::
+
+ $ qemu-system-aarch64 -M virt -cpu max,pmu=off,sve=on,sve128=on,sve256=on
+
+The example above disables the PMU and enables the first two SVE vector
+lengths for the `max` CPU type. Note, the `sve=on` isn't actually
+necessary, because, as we observed above with our probe of the `max` CPU
+type, `sve` is already on by default. Also, based on our probe of
+defaults, it would seem we need to disable many SVE vector lengths, rather
+than only enabling the two we want. This isn't the case, because, as
+disabling many SVE vector lengths would be quite verbose, the `sve<N>` CPU
+properties have special semantics (see "SVE CPU Property Parsing
+Semantics").
+
+SVE CPU Properties
+==================
+
+There are two types of SVE CPU properties: `sve` and `sve<N>`. The first
+is used to enable or disable the entire SVE feature, just as the `pmu`
+CPU property completely enables or disables the PMU. The second type
+is used to enable or disable specific vector lengths, where `N` is the
+number of bits of the length. The `sve<N>` CPU properties have special
+dependencies and constraints, see "SVE CPU Property Dependencies and
+Constraints" below. Additionally, as we want all supported vector lengths
+to be enabled by default, then, in order to avoid overly verbose command
+lines (command lines full of `sve<N>=off`, for all `N` not wanted), we
+provide the parsing semantics listed in "SVE CPU Property Parsing
+Semantics".
+
+SVE CPU Property Dependencies and Constraints
+---------------------------------------------
+
+ 1) At least one vector length must be enabled when `sve` is enabled.
+
+ 2) If a vector length `N` is enabled, then, when KVM is enabled, all
+ smaller, host supported vector lengths must also be enabled. If
+ KVM is not enabled, then only all the smaller, power-of-two vector
+ lengths must be enabled. E.g. with KVM if the host supports all
+ vector lengths up to 512-bits (128, 256, 384, 512), then if `sve512`
+ is enabled, the 128-bit vector length, 256-bit vector length, and
+ 384-bit vector length must also be enabled. Without KVM, the 384-bit
+ vector length would not be required.
+
+ 3) If KVM is enabled then only vector lengths that the host CPU type
+ support may be enabled. If SVE is not supported by the host, then
+ no `sve*` properties may be enabled.
+
+SVE CPU Property Parsing Semantics
+----------------------------------
+
+ 1) If SVE is disabled (`sve=off`), then which SVE vector lengths
+ are enabled or disabled is irrelevant to the guest, as the entire
+ SVE feature is disabled and that disables all vector lengths for
+ the guest. However QEMU will still track any `sve<N>` CPU
+ properties provided by the user. If later an `sve=on` is provided,
+ then the guest will get only the enabled lengths. If no `sve=on`
+ is provided and there are explicitly enabled vector lengths, then
+ an error is generated.
+
+ 2) If SVE is enabled (`sve=on`), but no `sve<N>` CPU properties are
+ provided, then all supported vector lengths are enabled, which when
+ KVM is not in use means including the non-power-of-two lengths, and,
+ when KVM is in use, it means all vector lengths supported by the host
+ processor.
+
+ 3) If SVE is enabled, then an error is generated when attempting to
+ disable the last enabled vector length (see constraint (1) of "SVE
+ CPU Property Dependencies and Constraints").
+
+ 4) If one or more vector lengths have been explicitly enabled and at
+ at least one of the dependency lengths of the maximum enabled length
+ has been explicitly disabled, then an error is generated (see
+ constraint (2) of "SVE CPU Property Dependencies and Constraints").
+
+ 5) When KVM is enabled, if the host does not support SVE, then an error
+ is generated when attempting to enable any `sve*` properties (see
+ constraint (3) of "SVE CPU Property Dependencies and Constraints").
+
+ 6) When KVM is enabled, if the host does support SVE, then an error is
+ generated when attempting to enable any vector lengths not supported
+ by the host (see constraint (3) of "SVE CPU Property Dependencies and
+ Constraints").
+
+ 7) If one or more `sve<N>` CPU properties are set `off`, but no `sve<N>`,
+ CPU properties are set `on`, then the specified vector lengths are
+ disabled but the default for any unspecified lengths remains enabled.
+ When KVM is not enabled, disabling a power-of-two vector length also
+ disables all vector lengths larger than the power-of-two length.
+ When KVM is enabled, then disabling any supported vector length also
+ disables all larger vector lengths (see constraint (2) of "SVE CPU
+ Property Dependencies and Constraints").
+
+ 8) If one or more `sve<N>` CPU properties are set to `on`, then they
+ are enabled and all unspecified lengths default to disabled, except
+ for the required lengths per constraint (2) of "SVE CPU Property
+ Dependencies and Constraints", which will even be auto-enabled if
+ they were not explicitly enabled.
+
+ 9) If SVE was disabled (`sve=off`), allowing all vector lengths to be
+ explicitly disabled (i.e. avoiding the error specified in (3) of
+ "SVE CPU Property Parsing Semantics"), then if later an `sve=on` is
+ provided an error will be generated. To avoid this error, one must
+ enable at least one vector length prior to enabling SVE.
+
+SVE CPU Property Examples
+-------------------------
+
+ 1) Disable SVE::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve=off
+
+ 2) Implicitly enable all vector lengths for the `max` CPU type::
+
+ $ qemu-system-aarch64 -M virt -cpu max
+
+ 3) When KVM is enabled, implicitly enable all host CPU supported vector
+ lengths with the `host` CPU type::
+
+ $ qemu-system-aarch64 -M virt,accel=kvm -cpu host
+
+ 4) Only enable the 128-bit vector length::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve128=on
+
+ 5) Disable the 512-bit vector length and all larger vector lengths,
+ since 512 is a power-of-two. This results in all the smaller,
+ uninitialized lengths (128, 256, and 384) defaulting to enabled::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve512=off
+
+ 6) Enable the 128-bit, 256-bit, and 512-bit vector lengths::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve128=on,sve256=on,sve512=on
+
+ 7) The same as (6), but since the 128-bit and 256-bit vector
+ lengths are required for the 512-bit vector length to be enabled,
+ then allow them to be auto-enabled::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve512=on
+
+ 8) Do the same as (7), but by first disabling SVE and then re-enabling it::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve512=on,sve=on
+
+ 9) Force errors regarding the last vector length::
+
+ $ qemu-system-aarch64 -M virt -cpu max,sve128=off
+ $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve128=off,sve=on
+
+SVE CPU Property Recommendations
+--------------------------------
+
+The examples in "SVE CPU Property Examples" exhibit many ways to select
+vector lengths which developers may find useful in order to avoid overly
+verbose command lines. However, the recommended way to select vector
+lengths is to explicitly enable each desired length. Therefore only
+example's (1), (4), and (6) exhibit recommended uses of the properties.
+
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index c264864..ef67249 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -786,6 +786,7 @@
info->secondary_cpu_reset_hook(cpu, info);
}
}
+ arm_rebuild_hflags(env);
}
}
diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h
index 3f0926c..ee76552 100644
--- a/include/qemu/bitops.h
+++ b/include/qemu/bitops.h
@@ -20,6 +20,7 @@
#define BITS_PER_LONG (sizeof (unsigned long) * BITS_PER_BYTE)
#define BIT(nr) (1UL << (nr))
+#define BIT_ULL(nr) (1ULL << (nr))
#define BIT_MASK(nr) (1UL << ((nr) % BITS_PER_LONG))
#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
#define BITS_TO_LONGS(nr) DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(long))
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 55310a6..0462322 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -212,7 +212,7 @@
##
{ 'struct': 'CpuModelExpansionInfo',
'data': { 'model': 'CpuModelInfo' },
- 'if': 'defined(TARGET_S390X) || defined(TARGET_I386)' }
+ 'if': 'defined(TARGET_S390X) || defined(TARGET_I386) || defined(TARGET_ARM)' }
##
# @query-cpu-model-expansion:
@@ -237,7 +237,7 @@
# query-cpu-model-expansion while using these is not advised.
#
# Some architectures may not support all expansion types. s390x supports
-# "full" and "static".
+# "full" and "static". Arm only supports "full".
#
# Returns: a CpuModelExpansionInfo. Returns an error if expanding CPU models is
# not supported, if the model cannot be expanded, if the model contains
@@ -251,7 +251,7 @@
'data': { 'type': 'CpuModelExpansionType',
'model': 'CpuModelInfo' },
'returns': 'CpuModelExpansionInfo',
- 'if': 'defined(TARGET_S390X) || defined(TARGET_I386)' }
+ 'if': 'defined(TARGET_S390X) || defined(TARGET_I386) || defined(TARGET_ARM)' }
##
# @CpuDefinitionInfo:
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index ab3e1a0..7a4ac93 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -200,7 +200,8 @@
env->cp15.cpacr_el1 = deposit64(env->cp15.cpacr_el1, 16, 2, 3);
env->cp15.cptr_el[3] |= CPTR_EZ;
/* with maximum vector length */
- env->vfp.zcr_el[1] = cpu->sve_max_vq - 1;
+ env->vfp.zcr_el[1] = cpu_isar_feature(aa64_sve, cpu) ?
+ cpu->sve_max_vq - 1 : 0;
env->vfp.zcr_el[2] = env->vfp.zcr_el[1];
env->vfp.zcr_el[3] = env->vfp.zcr_el[1];
/*
@@ -1197,6 +1198,19 @@
#endif
}
+void arm_cpu_finalize_features(ARMCPU *cpu, Error **errp)
+{
+ Error *local_err = NULL;
+
+ if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
+ arm_cpu_sve_finalize(cpu, &local_err);
+ if (local_err != NULL) {
+ error_propagate(errp, local_err);
+ return;
+ }
+ }
+}
+
static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
{
CPUState *cs = CPU(dev);
@@ -1253,6 +1267,12 @@
return;
}
+ arm_cpu_finalize_features(cpu, &local_err);
+ if (local_err != NULL) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
if (arm_feature(env, ARM_FEATURE_AARCH64) &&
cpu->has_vfp != cpu->has_neon) {
/*
@@ -2650,6 +2670,9 @@
ARMCPU *cpu = ARM_CPU(obj);
kvm_arm_set_cpu_features_from_host(cpu);
+ if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
+ aarch64_add_sve_properties(obj);
+ }
arm_cpu_post_init(obj);
}
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index d844ea2..e1a66a2 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -184,8 +184,13 @@
#ifdef TARGET_AARCH64
# define ARM_MAX_VQ 16
+void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp);
+uint32_t arm_cpu_vq_map_next_smaller(ARMCPU *cpu, uint32_t vq);
#else
# define ARM_MAX_VQ 1
+static inline void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp) { }
+static inline uint32_t arm_cpu_vq_map_next_smaller(ARMCPU *cpu, uint32_t vq)
+{ return 0; }
#endif
typedef struct ARMVectorReg {
@@ -918,6 +923,18 @@
/* Used to set the maximum vector length the cpu will support. */
uint32_t sve_max_vq;
+
+ /*
+ * In sve_vq_map each set bit is a supported vector length of
+ * (bit-number + 1) * 16 bytes, i.e. each bit number + 1 is the vector
+ * length in quadwords.
+ *
+ * While processing properties during initialization, corresponding
+ * sve_vq_init bits are set for bits in sve_vq_map that have been
+ * set by properties.
+ */
+ DECLARE_BITMAP(sve_vq_map, ARM_MAX_VQ);
+ DECLARE_BITMAP(sve_vq_init, ARM_MAX_VQ);
};
void arm_cpu_post_init(Object *obj);
@@ -960,11 +977,13 @@
void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq);
void aarch64_sve_change_el(CPUARMState *env, int old_el,
int new_el, bool el0_a64);
+void aarch64_add_sve_properties(Object *obj);
#else
static inline void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq) { }
static inline void aarch64_sve_change_el(CPUARMState *env, int o,
int n, bool a)
{ }
+static inline void aarch64_add_sve_properties(Object *obj) { }
#endif
#if !defined(CONFIG_TCG)
@@ -1837,6 +1856,8 @@
return (env->features & (1ULL << feature)) != 0;
}
+void arm_cpu_finalize_features(ARMCPU *cpu, Error **errp);
+
#if !defined(CONFIG_USER_ONLY)
/* Return true if exception levels below EL3 are in secure state,
* or would be following an exception return to that level.
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index d7f5bf6..68baf04 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -256,27 +256,357 @@
define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo);
}
-static void cpu_max_get_sve_vq(Object *obj, Visitor *v, const char *name,
- void *opaque, Error **errp)
+void arm_cpu_sve_finalize(ARMCPU *cpu, Error **errp)
{
- ARMCPU *cpu = ARM_CPU(obj);
- visit_type_uint32(v, name, &cpu->sve_max_vq, errp);
+ /*
+ * If any vector lengths are explicitly enabled with sve<N> properties,
+ * then all other lengths are implicitly disabled. If sve-max-vq is
+ * specified then it is the same as explicitly enabling all lengths
+ * up to and including the specified maximum, which means all larger
+ * lengths will be implicitly disabled. If no sve<N> properties
+ * are enabled and sve-max-vq is not specified, then all lengths not
+ * explicitly disabled will be enabled. Additionally, all power-of-two
+ * vector lengths less than the maximum enabled length will be
+ * automatically enabled and all vector lengths larger than the largest
+ * disabled power-of-two vector length will be automatically disabled.
+ * Errors are generated if the user provided input that interferes with
+ * any of the above. Finally, if SVE is not disabled, then at least one
+ * vector length must be enabled.
+ */
+ DECLARE_BITMAP(kvm_supported, ARM_MAX_VQ);
+ DECLARE_BITMAP(tmp, ARM_MAX_VQ);
+ uint32_t vq, max_vq = 0;
+
+ /* Collect the set of vector lengths supported by KVM. */
+ bitmap_zero(kvm_supported, ARM_MAX_VQ);
+ if (kvm_enabled() && kvm_arm_sve_supported(CPU(cpu))) {
+ kvm_arm_sve_get_vls(CPU(cpu), kvm_supported);
+ } else if (kvm_enabled()) {
+ assert(!cpu_isar_feature(aa64_sve, cpu));
+ }
+
+ /*
+ * Process explicit sve<N> properties.
+ * From the properties, sve_vq_map<N> implies sve_vq_init<N>.
+ * Check first for any sve<N> enabled.
+ */
+ if (!bitmap_empty(cpu->sve_vq_map, ARM_MAX_VQ)) {
+ max_vq = find_last_bit(cpu->sve_vq_map, ARM_MAX_VQ) + 1;
+
+ if (cpu->sve_max_vq && max_vq > cpu->sve_max_vq) {
+ error_setg(errp, "cannot enable sve%d", max_vq * 128);
+ error_append_hint(errp, "sve%d is larger than the maximum vector "
+ "length, sve-max-vq=%d (%d bits)\n",
+ max_vq * 128, cpu->sve_max_vq,
+ cpu->sve_max_vq * 128);
+ return;
+ }
+
+ if (kvm_enabled()) {
+ /*
+ * For KVM we have to automatically enable all supported unitialized
+ * lengths, even when the smaller lengths are not all powers-of-two.
+ */
+ bitmap_andnot(tmp, kvm_supported, cpu->sve_vq_init, max_vq);
+ bitmap_or(cpu->sve_vq_map, cpu->sve_vq_map, tmp, max_vq);
+ } else {
+ /* Propagate enabled bits down through required powers-of-two. */
+ for (vq = pow2floor(max_vq); vq >= 1; vq >>= 1) {
+ if (!test_bit(vq - 1, cpu->sve_vq_init)) {
+ set_bit(vq - 1, cpu->sve_vq_map);
+ }
+ }
+ }
+ } else if (cpu->sve_max_vq == 0) {
+ /*
+ * No explicit bits enabled, and no implicit bits from sve-max-vq.
+ */
+ if (!cpu_isar_feature(aa64_sve, cpu)) {
+ /* SVE is disabled and so are all vector lengths. Good. */
+ return;
+ }
+
+ if (kvm_enabled()) {
+ /* Disabling a supported length disables all larger lengths. */
+ for (vq = 1; vq <= ARM_MAX_VQ; ++vq) {
+ if (test_bit(vq - 1, cpu->sve_vq_init) &&
+ test_bit(vq - 1, kvm_supported)) {
+ break;
+ }
+ }
+ max_vq = vq <= ARM_MAX_VQ ? vq - 1 : ARM_MAX_VQ;
+ bitmap_andnot(cpu->sve_vq_map, kvm_supported,
+ cpu->sve_vq_init, max_vq);
+ if (max_vq == 0 || bitmap_empty(cpu->sve_vq_map, max_vq)) {
+ error_setg(errp, "cannot disable sve%d", vq * 128);
+ error_append_hint(errp, "Disabling sve%d results in all "
+ "vector lengths being disabled.\n",
+ vq * 128);
+ error_append_hint(errp, "With SVE enabled, at least one "
+ "vector length must be enabled.\n");
+ return;
+ }
+ } else {
+ /* Disabling a power-of-two disables all larger lengths. */
+ if (test_bit(0, cpu->sve_vq_init)) {
+ error_setg(errp, "cannot disable sve128");
+ error_append_hint(errp, "Disabling sve128 results in all "
+ "vector lengths being disabled.\n");
+ error_append_hint(errp, "With SVE enabled, at least one "
+ "vector length must be enabled.\n");
+ return;
+ }
+ for (vq = 2; vq <= ARM_MAX_VQ; vq <<= 1) {
+ if (test_bit(vq - 1, cpu->sve_vq_init)) {
+ break;
+ }
+ }
+ max_vq = vq <= ARM_MAX_VQ ? vq - 1 : ARM_MAX_VQ;
+ bitmap_complement(cpu->sve_vq_map, cpu->sve_vq_init, max_vq);
+ }
+
+ max_vq = find_last_bit(cpu->sve_vq_map, max_vq) + 1;
+ }
+
+ /*
+ * Process the sve-max-vq property.
+ * Note that we know from the above that no bit above
+ * sve-max-vq is currently set.
+ */
+ if (cpu->sve_max_vq != 0) {
+ max_vq = cpu->sve_max_vq;
+
+ if (!test_bit(max_vq - 1, cpu->sve_vq_map) &&
+ test_bit(max_vq - 1, cpu->sve_vq_init)) {
+ error_setg(errp, "cannot disable sve%d", max_vq * 128);
+ error_append_hint(errp, "The maximum vector length must be "
+ "enabled, sve-max-vq=%d (%d bits)\n",
+ max_vq, max_vq * 128);
+ return;
+ }
+
+ /* Set all bits not explicitly set within sve-max-vq. */
+ bitmap_complement(tmp, cpu->sve_vq_init, max_vq);
+ bitmap_or(cpu->sve_vq_map, cpu->sve_vq_map, tmp, max_vq);
+ }
+
+ /*
+ * We should know what max-vq is now. Also, as we're done
+ * manipulating sve-vq-map, we ensure any bits above max-vq
+ * are clear, just in case anybody looks.
+ */
+ assert(max_vq != 0);
+ bitmap_clear(cpu->sve_vq_map, max_vq, ARM_MAX_VQ - max_vq);
+
+ if (kvm_enabled()) {
+ /* Ensure the set of lengths matches what KVM supports. */
+ bitmap_xor(tmp, cpu->sve_vq_map, kvm_supported, max_vq);
+ if (!bitmap_empty(tmp, max_vq)) {
+ vq = find_last_bit(tmp, max_vq) + 1;
+ if (test_bit(vq - 1, cpu->sve_vq_map)) {
+ if (cpu->sve_max_vq) {
+ error_setg(errp, "cannot set sve-max-vq=%d",
+ cpu->sve_max_vq);
+ error_append_hint(errp, "This KVM host does not support "
+ "the vector length %d-bits.\n",
+ vq * 128);
+ error_append_hint(errp, "It may not be possible to use "
+ "sve-max-vq with this KVM host. Try "
+ "using only sve<N> properties.\n");
+ } else {
+ error_setg(errp, "cannot enable sve%d", vq * 128);
+ error_append_hint(errp, "This KVM host does not support "
+ "the vector length %d-bits.\n",
+ vq * 128);
+ }
+ } else {
+ error_setg(errp, "cannot disable sve%d", vq * 128);
+ error_append_hint(errp, "The KVM host requires all "
+ "supported vector lengths smaller "
+ "than %d bits to also be enabled.\n",
+ max_vq * 128);
+ }
+ return;
+ }
+ } else {
+ /* Ensure all required powers-of-two are enabled. */
+ for (vq = pow2floor(max_vq); vq >= 1; vq >>= 1) {
+ if (!test_bit(vq - 1, cpu->sve_vq_map)) {
+ error_setg(errp, "cannot disable sve%d", vq * 128);
+ error_append_hint(errp, "sve%d is required as it "
+ "is a power-of-two length smaller than "
+ "the maximum, sve%d\n",
+ vq * 128, max_vq * 128);
+ return;
+ }
+ }
+ }
+
+ /*
+ * Now that we validated all our vector lengths, the only question
+ * left to answer is if we even want SVE at all.
+ */
+ if (!cpu_isar_feature(aa64_sve, cpu)) {
+ error_setg(errp, "cannot enable sve%d", max_vq * 128);
+ error_append_hint(errp, "SVE must be enabled to enable vector "
+ "lengths.\n");
+ error_append_hint(errp, "Add sve=on to the CPU property list.\n");
+ return;
+ }
+
+ /* From now on sve_max_vq is the actual maximum supported length. */
+ cpu->sve_max_vq = max_vq;
}
-static void cpu_max_set_sve_vq(Object *obj, Visitor *v, const char *name,
- void *opaque, Error **errp)
+uint32_t arm_cpu_vq_map_next_smaller(ARMCPU *cpu, uint32_t vq)
+{
+ uint32_t bitnum;
+
+ /*
+ * We allow vq == ARM_MAX_VQ + 1 to be input because the caller may want
+ * to find the maximum vq enabled, which may be ARM_MAX_VQ, but this
+ * function always returns the next smaller than the input.
+ */
+ assert(vq && vq <= ARM_MAX_VQ + 1);
+
+ bitnum = find_last_bit(cpu->sve_vq_map, vq - 1);
+ return bitnum == vq - 1 ? 0 : bitnum + 1;
+}
+
+static void cpu_max_get_sve_max_vq(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ ARMCPU *cpu = ARM_CPU(obj);
+ uint32_t value;
+
+ /* All vector lengths are disabled when SVE is off. */
+ if (!cpu_isar_feature(aa64_sve, cpu)) {
+ value = 0;
+ } else {
+ value = cpu->sve_max_vq;
+ }
+ visit_type_uint32(v, name, &value, errp);
+}
+
+static void cpu_max_set_sve_max_vq(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
{
ARMCPU *cpu = ARM_CPU(obj);
Error *err = NULL;
+ uint32_t max_vq;
- visit_type_uint32(v, name, &cpu->sve_max_vq, &err);
-
- if (!err && (cpu->sve_max_vq == 0 || cpu->sve_max_vq > ARM_MAX_VQ)) {
- error_setg(&err, "unsupported SVE vector length");
- error_append_hint(&err, "Valid sve-max-vq in range [1-%d]\n",
- ARM_MAX_VQ);
+ visit_type_uint32(v, name, &max_vq, &err);
+ if (err) {
+ error_propagate(errp, err);
+ return;
}
- error_propagate(errp, err);
+
+ if (kvm_enabled() && !kvm_arm_sve_supported(CPU(cpu))) {
+ error_setg(errp, "cannot set sve-max-vq");
+ error_append_hint(errp, "SVE not supported by KVM on this host\n");
+ return;
+ }
+
+ if (max_vq == 0 || max_vq > ARM_MAX_VQ) {
+ error_setg(errp, "unsupported SVE vector length");
+ error_append_hint(errp, "Valid sve-max-vq in range [1-%d]\n",
+ ARM_MAX_VQ);
+ return;
+ }
+
+ cpu->sve_max_vq = max_vq;
+}
+
+static void cpu_arm_get_sve_vq(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ ARMCPU *cpu = ARM_CPU(obj);
+ uint32_t vq = atoi(&name[3]) / 128;
+ bool value;
+
+ /* All vector lengths are disabled when SVE is off. */
+ if (!cpu_isar_feature(aa64_sve, cpu)) {
+ value = false;
+ } else {
+ value = test_bit(vq - 1, cpu->sve_vq_map);
+ }
+ visit_type_bool(v, name, &value, errp);
+}
+
+static void cpu_arm_set_sve_vq(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ ARMCPU *cpu = ARM_CPU(obj);
+ uint32_t vq = atoi(&name[3]) / 128;
+ Error *err = NULL;
+ bool value;
+
+ visit_type_bool(v, name, &value, &err);
+ if (err) {
+ error_propagate(errp, err);
+ return;
+ }
+
+ if (value && kvm_enabled() && !kvm_arm_sve_supported(CPU(cpu))) {
+ error_setg(errp, "cannot enable %s", name);
+ error_append_hint(errp, "SVE not supported by KVM on this host\n");
+ return;
+ }
+
+ if (value) {
+ set_bit(vq - 1, cpu->sve_vq_map);
+ } else {
+ clear_bit(vq - 1, cpu->sve_vq_map);
+ }
+ set_bit(vq - 1, cpu->sve_vq_init);
+}
+
+static void cpu_arm_get_sve(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ ARMCPU *cpu = ARM_CPU(obj);
+ bool value = cpu_isar_feature(aa64_sve, cpu);
+
+ visit_type_bool(v, name, &value, errp);
+}
+
+static void cpu_arm_set_sve(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ ARMCPU *cpu = ARM_CPU(obj);
+ Error *err = NULL;
+ bool value;
+ uint64_t t;
+
+ visit_type_bool(v, name, &value, &err);
+ if (err) {
+ error_propagate(errp, err);
+ return;
+ }
+
+ if (value && kvm_enabled() && !kvm_arm_sve_supported(CPU(cpu))) {
+ error_setg(errp, "'sve' feature not supported by KVM on this host");
+ return;
+ }
+
+ t = cpu->isar.id_aa64pfr0;
+ t = FIELD_DP64(t, ID_AA64PFR0, SVE, value);
+ cpu->isar.id_aa64pfr0 = t;
+}
+
+void aarch64_add_sve_properties(Object *obj)
+{
+ uint32_t vq;
+
+ object_property_add(obj, "sve", "bool", cpu_arm_get_sve,
+ cpu_arm_set_sve, NULL, NULL, &error_fatal);
+
+ for (vq = 1; vq <= ARM_MAX_VQ; ++vq) {
+ char name[8];
+ sprintf(name, "sve%d", vq * 128);
+ object_property_add(obj, name, "bool", cpu_arm_get_sve_vq,
+ cpu_arm_set_sve_vq, NULL, NULL, &error_fatal);
+ }
}
/* -cpu max: if KVM is enabled, like -cpu host (best possible with this host);
@@ -389,11 +719,11 @@
cpu->ctr = 0x80038003; /* 32 byte I and D cacheline size, VIPT icache */
cpu->dcz_blocksize = 7; /* 512 bytes */
#endif
-
- cpu->sve_max_vq = ARM_MAX_VQ;
- object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_vq,
- cpu_max_set_sve_vq, NULL, NULL, &error_fatal);
}
+
+ aarch64_add_sve_properties(obj);
+ object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_max_vq,
+ cpu_max_set_sve_max_vq, NULL, NULL, &error_fatal);
}
struct ARMCPUInfo {
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 63815fc..be67e2c 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5361,6 +5361,13 @@
return 0;
}
+static uint32_t sve_zcr_get_valid_len(ARMCPU *cpu, uint32_t start_len)
+{
+ uint32_t start_vq = (start_len & 0xf) + 1;
+
+ return arm_cpu_vq_map_next_smaller(cpu, start_vq + 1) - 1;
+}
+
/*
* Given that SVE is enabled, return the vector length for EL.
*/
@@ -5378,7 +5385,8 @@
if (arm_feature(env, ARM_FEATURE_EL3)) {
zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[3]);
}
- return zcr_len;
+
+ return sve_zcr_get_valid_len(cpu, zcr_len);
}
static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index b473c63..5b82cef 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -51,6 +51,11 @@
return kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_INIT, &init);
}
+int kvm_arm_vcpu_finalize(CPUState *cs, int feature)
+{
+ return kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_FINALIZE, &feature);
+}
+
void kvm_arm_init_serror_injection(CPUState *cs)
{
cap_has_inject_serror_esr = kvm_check_extension(cs->kvm_state,
@@ -61,7 +66,7 @@
int *fdarray,
struct kvm_vcpu_init *init)
{
- int ret, kvmfd = -1, vmfd = -1, cpufd = -1;
+ int ret = 0, kvmfd = -1, vmfd = -1, cpufd = -1;
kvmfd = qemu_open("/dev/kvm", O_RDWR);
if (kvmfd < 0) {
@@ -81,7 +86,14 @@
goto finish;
}
- ret = ioctl(vmfd, KVM_ARM_PREFERRED_TARGET, init);
+ if (init->target == -1) {
+ struct kvm_vcpu_init preferred;
+
+ ret = ioctl(vmfd, KVM_ARM_PREFERRED_TARGET, &preferred);
+ if (!ret) {
+ init->target = preferred.target;
+ }
+ }
if (ret >= 0) {
ret = ioctl(cpufd, KVM_ARM_VCPU_INIT, init);
if (ret < 0) {
@@ -93,10 +105,12 @@
* creating one kind of guest CPU which is its preferred
* CPU type.
*/
+ struct kvm_vcpu_init try;
+
while (*cpus_to_try != QEMU_KVM_ARM_TARGET_NONE) {
- init->target = *cpus_to_try++;
- memset(init->features, 0, sizeof(init->features));
- ret = ioctl(cpufd, KVM_ARM_VCPU_INIT, init);
+ try.target = *cpus_to_try++;
+ memcpy(try.features, init->features, sizeof(init->features));
+ ret = ioctl(cpufd, KVM_ARM_VCPU_INIT, &try);
if (ret >= 0) {
break;
}
@@ -104,6 +118,7 @@
if (ret < 0) {
goto err;
}
+ init->target = try.target;
} else {
/* Treat a NULL cpus_to_try argument the same as an empty
* list, which means we will fail the call since this must
diff --git a/target/arm/kvm32.c b/target/arm/kvm32.c
index 2451a2d..32bf8d6 100644
--- a/target/arm/kvm32.c
+++ b/target/arm/kvm32.c
@@ -53,7 +53,11 @@
QEMU_KVM_ARM_TARGET_CORTEX_A15,
QEMU_KVM_ARM_TARGET_NONE
};
- struct kvm_vcpu_init init;
+ /*
+ * target = -1 informs kvm_arm_create_scratch_host_vcpu()
+ * to use the preferred target
+ */
+ struct kvm_vcpu_init init = { .target = -1, };
if (!kvm_arm_create_scratch_host_vcpu(cpus_to_try, fdarray, &init)) {
return false;
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index 28f6db5..876184b 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -488,7 +488,9 @@
* and then query that CPU for the relevant ID registers.
*/
int fdarray[3];
+ bool sve_supported;
uint64_t features = 0;
+ uint64_t t;
int err;
/* Old kernels may not know about the PREFERRED_TARGET ioctl: however
@@ -502,7 +504,11 @@
KVM_ARM_TARGET_CORTEX_A57,
QEMU_KVM_ARM_TARGET_NONE
};
- struct kvm_vcpu_init init;
+ /*
+ * target = -1 informs kvm_arm_create_scratch_host_vcpu()
+ * to use the preferred target
+ */
+ struct kvm_vcpu_init init = { .target = -1, };
if (!kvm_arm_create_scratch_host_vcpu(cpus_to_try, fdarray, &init)) {
return false;
@@ -574,13 +580,23 @@
ARM64_SYS_REG(3, 0, 0, 3, 2));
}
+ sve_supported = ioctl(fdarray[0], KVM_CHECK_EXTENSION, KVM_CAP_ARM_SVE) > 0;
+
kvm_arm_destroy_scratch_host_vcpu(fdarray);
if (err < 0) {
return false;
}
- /* We can assume any KVM supporting CPU is at least a v8
+ /* Add feature bits that can't appear until after VCPU init. */
+ if (sve_supported) {
+ t = ahcf->isar.id_aa64pfr0;
+ t = FIELD_DP64(t, ID_AA64PFR0, SVE, 1);
+ ahcf->isar.id_aa64pfr0 = t;
+ }
+
+ /*
+ * We can assume any KVM supporting CPU is at least a v8
* with VFPv4+Neon; this in turn implies most of the other
* feature bits.
*/
@@ -602,6 +618,107 @@
return kvm_check_extension(s, KVM_CAP_ARM_EL1_32BIT);
}
+bool kvm_arm_sve_supported(CPUState *cpu)
+{
+ KVMState *s = KVM_STATE(current_machine->accelerator);
+
+ return kvm_check_extension(s, KVM_CAP_ARM_SVE);
+}
+
+QEMU_BUILD_BUG_ON(KVM_ARM64_SVE_VQ_MIN != 1);
+
+void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map)
+{
+ /* Only call this function if kvm_arm_sve_supported() returns true. */
+ static uint64_t vls[KVM_ARM64_SVE_VLS_WORDS];
+ static bool probed;
+ uint32_t vq = 0;
+ int i, j;
+
+ bitmap_clear(map, 0, ARM_MAX_VQ);
+
+ /*
+ * KVM ensures all host CPUs support the same set of vector lengths.
+ * So we only need to create the scratch VCPUs once and then cache
+ * the results.
+ */
+ if (!probed) {
+ struct kvm_vcpu_init init = {
+ .target = -1,
+ .features[0] = (1 << KVM_ARM_VCPU_SVE),
+ };
+ struct kvm_one_reg reg = {
+ .id = KVM_REG_ARM64_SVE_VLS,
+ .addr = (uint64_t)&vls[0],
+ };
+ int fdarray[3], ret;
+
+ probed = true;
+
+ if (!kvm_arm_create_scratch_host_vcpu(NULL, fdarray, &init)) {
+ error_report("failed to create scratch VCPU with SVE enabled");
+ abort();
+ }
+ ret = ioctl(fdarray[2], KVM_GET_ONE_REG, ®);
+ kvm_arm_destroy_scratch_host_vcpu(fdarray);
+ if (ret) {
+ error_report("failed to get KVM_REG_ARM64_SVE_VLS: %s",
+ strerror(errno));
+ abort();
+ }
+
+ for (i = KVM_ARM64_SVE_VLS_WORDS - 1; i >= 0; --i) {
+ if (vls[i]) {
+ vq = 64 - clz64(vls[i]) + i * 64;
+ break;
+ }
+ }
+ if (vq > ARM_MAX_VQ) {
+ warn_report("KVM supports vector lengths larger than "
+ "QEMU can enable");
+ }
+ }
+
+ for (i = 0; i < KVM_ARM64_SVE_VLS_WORDS; ++i) {
+ if (!vls[i]) {
+ continue;
+ }
+ for (j = 1; j <= 64; ++j) {
+ vq = j + i * 64;
+ if (vq > ARM_MAX_VQ) {
+ return;
+ }
+ if (vls[i] & (1UL << (j - 1))) {
+ set_bit(vq - 1, map);
+ }
+ }
+ }
+}
+
+static int kvm_arm_sve_set_vls(CPUState *cs)
+{
+ uint64_t vls[KVM_ARM64_SVE_VLS_WORDS] = {0};
+ struct kvm_one_reg reg = {
+ .id = KVM_REG_ARM64_SVE_VLS,
+ .addr = (uint64_t)&vls[0],
+ };
+ ARMCPU *cpu = ARM_CPU(cs);
+ uint32_t vq;
+ int i, j;
+
+ assert(cpu->sve_max_vq <= KVM_ARM64_SVE_VQ_MAX);
+
+ for (vq = 1; vq <= cpu->sve_max_vq; ++vq) {
+ if (test_bit(vq - 1, cpu->sve_vq_map)) {
+ i = (vq - 1) / 64;
+ j = (vq - 1) % 64;
+ vls[i] |= 1UL << j;
+ }
+ }
+
+ return kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
+}
+
#define ARM_CPU_ID_MPIDR 3, 0, 0, 0, 5
int kvm_arch_init_vcpu(CPUState *cs)
@@ -613,7 +730,7 @@
if (cpu->kvm_target == QEMU_KVM_ARM_TARGET_NONE ||
!object_dynamic_cast(OBJECT(cpu), TYPE_AARCH64_CPU)) {
- fprintf(stderr, "KVM is not supported for this guest CPU type\n");
+ error_report("KVM is not supported for this guest CPU type");
return -EINVAL;
}
@@ -630,13 +747,17 @@
cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_EL1_32BIT;
}
if (!kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PMU_V3)) {
- cpu->has_pmu = false;
+ cpu->has_pmu = false;
}
if (cpu->has_pmu) {
cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_PMU_V3;
} else {
unset_feature(&env->features, ARM_FEATURE_PMU);
}
+ if (cpu_isar_feature(aa64_sve, cpu)) {
+ assert(kvm_arm_sve_supported(cs));
+ cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_SVE;
+ }
/* Do KVM_ARM_VCPU_INIT ioctl */
ret = kvm_arm_vcpu_init(cs);
@@ -644,6 +765,17 @@
return ret;
}
+ if (cpu_isar_feature(aa64_sve, cpu)) {
+ ret = kvm_arm_sve_set_vls(cs);
+ if (ret) {
+ return ret;
+ }
+ ret = kvm_arm_vcpu_finalize(cs, KVM_ARM_VCPU_SVE);
+ if (ret) {
+ return ret;
+ }
+ }
+
/*
* When KVM is in use, PSCI is emulated in-kernel and not by qemu.
* Currently KVM has its own idea about MPIDR assignment, so we
@@ -671,11 +803,12 @@
bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t regidx)
{
/* Return true if the regidx is a register we should synchronize
- * via the cpreg_tuples array (ie is not a core reg we sync by
- * hand in kvm_arch_get/put_registers())
+ * via the cpreg_tuples array (ie is not a core or sve reg that
+ * we sync by hand in kvm_arch_get/put_registers())
*/
switch (regidx & KVM_REG_ARM_COPROC_MASK) {
case KVM_REG_ARM_CORE:
+ case KVM_REG_ARM64_SVE:
return false;
default:
return true;
@@ -721,10 +854,8 @@
static int kvm_arch_put_fpsimd(CPUState *cs)
{
- ARMCPU *cpu = ARM_CPU(cs);
- CPUARMState *env = &cpu->env;
+ CPUARMState *env = &ARM_CPU(cs)->env;
struct kvm_one_reg reg;
- uint32_t fpr;
int i, ret;
for (i = 0; i < 32; i++) {
@@ -742,17 +873,73 @@
}
}
- reg.addr = (uintptr_t)(&fpr);
- fpr = vfp_get_fpsr(env);
- reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr);
- ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
- if (ret) {
- return ret;
+ return 0;
+}
+
+/*
+ * SVE registers are encoded in KVM's memory in an endianness-invariant format.
+ * The byte at offset i from the start of the in-memory representation contains
+ * the bits [(7 + 8 * i) : (8 * i)] of the register value. As this means the
+ * lowest offsets are stored in the lowest memory addresses, then that nearly
+ * matches QEMU's representation, which is to use an array of host-endian
+ * uint64_t's, where the lower offsets are at the lower indices. To complete
+ * the translation we just need to byte swap the uint64_t's on big-endian hosts.
+ */
+static uint64_t *sve_bswap64(uint64_t *dst, uint64_t *src, int nr)
+{
+#ifdef HOST_WORDS_BIGENDIAN
+ int i;
+
+ for (i = 0; i < nr; ++i) {
+ dst[i] = bswap64(src[i]);
}
- reg.addr = (uintptr_t)(&fpr);
- fpr = vfp_get_fpcr(env);
- reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr);
+ return dst;
+#else
+ return src;
+#endif
+}
+
+/*
+ * KVM SVE registers come in slices where ZREGs have a slice size of 2048 bits
+ * and PREGS and the FFR have a slice size of 256 bits. However we simply hard
+ * code the slice index to zero for now as it's unlikely we'll need more than
+ * one slice for quite some time.
+ */
+static int kvm_arch_put_sve(CPUState *cs)
+{
+ ARMCPU *cpu = ARM_CPU(cs);
+ CPUARMState *env = &cpu->env;
+ uint64_t tmp[ARM_MAX_VQ * 2];
+ uint64_t *r;
+ struct kvm_one_reg reg;
+ int n, ret;
+
+ for (n = 0; n < KVM_ARM64_SVE_NUM_ZREGS; ++n) {
+ r = sve_bswap64(tmp, &env->vfp.zregs[n].d[0], cpu->sve_max_vq * 2);
+ reg.addr = (uintptr_t)r;
+ reg.id = KVM_REG_ARM64_SVE_ZREG(n, 0);
+ ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+ }
+
+ for (n = 0; n < KVM_ARM64_SVE_NUM_PREGS; ++n) {
+ r = sve_bswap64(tmp, r = &env->vfp.pregs[n].p[0],
+ DIV_ROUND_UP(cpu->sve_max_vq * 2, 8));
+ reg.addr = (uintptr_t)r;
+ reg.id = KVM_REG_ARM64_SVE_PREG(n, 0);
+ ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+ }
+
+ r = sve_bswap64(tmp, &env->vfp.pregs[FFR_PRED_NUM].p[0],
+ DIV_ROUND_UP(cpu->sve_max_vq * 2, 8));
+ reg.addr = (uintptr_t)r;
+ reg.id = KVM_REG_ARM64_SVE_FFR(0);
ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
if (ret) {
return ret;
@@ -765,6 +952,7 @@
{
struct kvm_one_reg reg;
uint64_t val;
+ uint32_t fpr;
int i, ret;
unsigned int el;
@@ -855,7 +1043,27 @@
}
}
- ret = kvm_arch_put_fpsimd(cs);
+ if (cpu_isar_feature(aa64_sve, cpu)) {
+ ret = kvm_arch_put_sve(cs);
+ } else {
+ ret = kvm_arch_put_fpsimd(cs);
+ }
+ if (ret) {
+ return ret;
+ }
+
+ reg.addr = (uintptr_t)(&fpr);
+ fpr = vfp_get_fpsr(env);
+ reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr);
+ ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+
+ reg.addr = (uintptr_t)(&fpr);
+ fpr = vfp_get_fpcr(env);
+ reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr);
+ ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®);
if (ret) {
return ret;
}
@@ -878,10 +1086,8 @@
static int kvm_arch_get_fpsimd(CPUState *cs)
{
- ARMCPU *cpu = ARM_CPU(cs);
- CPUARMState *env = &cpu->env;
+ CPUARMState *env = &ARM_CPU(cs)->env;
struct kvm_one_reg reg;
- uint32_t fpr;
int i, ret;
for (i = 0; i < 32; i++) {
@@ -899,21 +1105,53 @@
}
}
- reg.addr = (uintptr_t)(&fpr);
- reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr);
- ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®);
- if (ret) {
- return ret;
- }
- vfp_set_fpsr(env, fpr);
+ return 0;
+}
- reg.addr = (uintptr_t)(&fpr);
- reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr);
+/*
+ * KVM SVE registers come in slices where ZREGs have a slice size of 2048 bits
+ * and PREGS and the FFR have a slice size of 256 bits. However we simply hard
+ * code the slice index to zero for now as it's unlikely we'll need more than
+ * one slice for quite some time.
+ */
+static int kvm_arch_get_sve(CPUState *cs)
+{
+ ARMCPU *cpu = ARM_CPU(cs);
+ CPUARMState *env = &cpu->env;
+ struct kvm_one_reg reg;
+ uint64_t *r;
+ int n, ret;
+
+ for (n = 0; n < KVM_ARM64_SVE_NUM_ZREGS; ++n) {
+ r = &env->vfp.zregs[n].d[0];
+ reg.addr = (uintptr_t)r;
+ reg.id = KVM_REG_ARM64_SVE_ZREG(n, 0);
+ ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+ sve_bswap64(r, r, cpu->sve_max_vq * 2);
+ }
+
+ for (n = 0; n < KVM_ARM64_SVE_NUM_PREGS; ++n) {
+ r = &env->vfp.pregs[n].p[0];
+ reg.addr = (uintptr_t)r;
+ reg.id = KVM_REG_ARM64_SVE_PREG(n, 0);
+ ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+ sve_bswap64(r, r, DIV_ROUND_UP(cpu->sve_max_vq * 2, 8));
+ }
+
+ r = &env->vfp.pregs[FFR_PRED_NUM].p[0];
+ reg.addr = (uintptr_t)r;
+ reg.id = KVM_REG_ARM64_SVE_FFR(0);
ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®);
if (ret) {
return ret;
}
- vfp_set_fpcr(env, fpr);
+ sve_bswap64(r, r, DIV_ROUND_UP(cpu->sve_max_vq * 2, 8));
return 0;
}
@@ -923,6 +1161,7 @@
struct kvm_one_reg reg;
uint64_t val;
unsigned int el;
+ uint32_t fpr;
int i, ret;
ARMCPU *cpu = ARM_CPU(cs);
@@ -1012,11 +1251,31 @@
env->spsr = env->banked_spsr[i];
}
- ret = kvm_arch_get_fpsimd(cs);
+ if (cpu_isar_feature(aa64_sve, cpu)) {
+ ret = kvm_arch_get_sve(cs);
+ } else {
+ ret = kvm_arch_get_fpsimd(cs);
+ }
if (ret) {
return ret;
}
+ reg.addr = (uintptr_t)(&fpr);
+ reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr);
+ ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+ vfp_set_fpsr(env, fpr);
+
+ reg.addr = (uintptr_t)(&fpr);
+ reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr);
+ ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®);
+ if (ret) {
+ return ret;
+ }
+ vfp_set_fpcr(env, fpr);
+
ret = kvm_get_vcpu_events(cpu);
if (ret) {
return ret;
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index b4e1945..8e14d40 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -28,6 +28,20 @@
int kvm_arm_vcpu_init(CPUState *cs);
/**
+ * kvm_arm_vcpu_finalize
+ * @cs: CPUState
+ * @feature: int
+ *
+ * Finalizes the configuration of the specified VCPU feature by
+ * invoking the KVM_ARM_VCPU_FINALIZE ioctl. Features requiring
+ * this are documented in the "KVM_ARM_VCPU_FINALIZE" section of
+ * KVM's API documentation.
+ *
+ * Returns: 0 if success else < 0 error code
+ */
+int kvm_arm_vcpu_finalize(CPUState *cs, int feature);
+
+/**
* kvm_arm_register_device:
* @mr: memory region for this device
* @devid: the KVM device ID
@@ -199,6 +213,17 @@
bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf);
/**
+ * kvm_arm_sve_get_vls:
+ * @cs: CPUState
+ * @map: bitmap to fill in
+ *
+ * Get all the SVE vector lengths supported by the KVM host, setting
+ * the bits corresponding to their length in quadwords minus one
+ * (vq - 1) in @map up to ARM_MAX_VQ.
+ */
+void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map);
+
+/**
* kvm_arm_set_cpu_features_from_host:
* @cpu: ARMCPU to set the features for
*
@@ -226,6 +251,14 @@
bool kvm_arm_pmu_supported(CPUState *cs);
/**
+ * bool kvm_arm_sve_supported:
+ * @cs: CPUState
+ *
+ * Returns true if the KVM VCPU can enable SVE and false otherwise.
+ */
+bool kvm_arm_sve_supported(CPUState *cs);
+
+/**
* kvm_arm_get_max_vm_ipa_size - Returns the number of bits in the
* IPA address space supported by KVM
*
@@ -276,6 +309,11 @@
return false;
}
+static inline bool kvm_arm_sve_supported(CPUState *cs)
+{
+ return false;
+}
+
static inline int kvm_arm_get_max_vm_ipa_size(MachineState *ms)
{
return -ENOENT;
@@ -289,6 +327,7 @@
static inline void kvm_arm_pmu_set_irq(CPUState *cs, int irq) {}
static inline void kvm_arm_pmu_init(CPUState *cs) {}
+static inline void kvm_arm_sve_get_vls(CPUState *cs, unsigned long *map) {}
#endif
static inline const char *gic_class_name(void)
diff --git a/target/arm/monitor.c b/target/arm/monitor.c
index 6457c3c..fa054f8 100644
--- a/target/arm/monitor.c
+++ b/target/arm/monitor.c
@@ -21,8 +21,16 @@
*/
#include "qemu/osdep.h"
+#include "hw/boards.h"
#include "kvm_arm.h"
+#include "qapi/error.h"
+#include "qapi/visitor.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qapi/qapi-commands-machine-target.h"
#include "qapi/qapi-commands-misc-target.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qdict.h"
+#include "qom/qom-qobject.h"
static GICCapability *gic_cap_new(int version)
{
@@ -81,3 +89,153 @@
return head;
}
+
+QEMU_BUILD_BUG_ON(ARM_MAX_VQ > 16);
+
+/*
+ * These are cpu model features we want to advertise. The order here
+ * matters as this is the order in which qmp_query_cpu_model_expansion
+ * will attempt to set them. If there are dependencies between features,
+ * then the order that considers those dependencies must be used.
+ */
+static const char *cpu_model_advertised_features[] = {
+ "aarch64", "pmu", "sve",
+ "sve128", "sve256", "sve384", "sve512",
+ "sve640", "sve768", "sve896", "sve1024", "sve1152", "sve1280",
+ "sve1408", "sve1536", "sve1664", "sve1792", "sve1920", "sve2048",
+ NULL
+};
+
+CpuModelExpansionInfo *qmp_query_cpu_model_expansion(CpuModelExpansionType type,
+ CpuModelInfo *model,
+ Error **errp)
+{
+ CpuModelExpansionInfo *expansion_info;
+ const QDict *qdict_in = NULL;
+ QDict *qdict_out;
+ ObjectClass *oc;
+ Object *obj;
+ const char *name;
+ int i;
+
+ if (type != CPU_MODEL_EXPANSION_TYPE_FULL) {
+ error_setg(errp, "The requested expansion type is not supported");
+ return NULL;
+ }
+
+ if (!kvm_enabled() && !strcmp(model->name, "host")) {
+ error_setg(errp, "The CPU type '%s' requires KVM", model->name);
+ return NULL;
+ }
+
+ oc = cpu_class_by_name(TYPE_ARM_CPU, model->name);
+ if (!oc) {
+ error_setg(errp, "The CPU type '%s' is not a recognized ARM CPU type",
+ model->name);
+ return NULL;
+ }
+
+ if (kvm_enabled()) {
+ const char *cpu_type = current_machine->cpu_type;
+ int len = strlen(cpu_type) - strlen(ARM_CPU_TYPE_SUFFIX);
+ bool supported = false;
+
+ if (!strcmp(model->name, "host") || !strcmp(model->name, "max")) {
+ /* These are kvmarm's recommended cpu types */
+ supported = true;
+ } else if (strlen(model->name) == len &&
+ !strncmp(model->name, cpu_type, len)) {
+ /* KVM is enabled and we're using this type, so it works. */
+ supported = true;
+ }
+ if (!supported) {
+ error_setg(errp, "We cannot guarantee the CPU type '%s' works "
+ "with KVM on this host", model->name);
+ return NULL;
+ }
+ }
+
+ if (model->props) {
+ qdict_in = qobject_to(QDict, model->props);
+ if (!qdict_in) {
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, "props", "dict");
+ return NULL;
+ }
+ }
+
+ obj = object_new(object_class_get_name(oc));
+
+ if (qdict_in) {
+ Visitor *visitor;
+ Error *err = NULL;
+
+ visitor = qobject_input_visitor_new(model->props);
+ visit_start_struct(visitor, NULL, NULL, 0, &err);
+ if (err) {
+ visit_free(visitor);
+ object_unref(obj);
+ error_propagate(errp, err);
+ return NULL;
+ }
+
+ i = 0;
+ while ((name = cpu_model_advertised_features[i++]) != NULL) {
+ if (qdict_get(qdict_in, name)) {
+ object_property_set(obj, visitor, name, &err);
+ if (err) {
+ break;
+ }
+ }
+ }
+
+ if (!err) {
+ visit_check_struct(visitor, &err);
+ }
+ if (!err) {
+ arm_cpu_finalize_features(ARM_CPU(obj), &err);
+ }
+ visit_end_struct(visitor, NULL);
+ visit_free(visitor);
+ if (err) {
+ object_unref(obj);
+ error_propagate(errp, err);
+ return NULL;
+ }
+ } else {
+ Error *err = NULL;
+ arm_cpu_finalize_features(ARM_CPU(obj), &err);
+ assert(err == NULL);
+ }
+
+ expansion_info = g_new0(CpuModelExpansionInfo, 1);
+ expansion_info->model = g_malloc0(sizeof(*expansion_info->model));
+ expansion_info->model->name = g_strdup(model->name);
+
+ qdict_out = qdict_new();
+
+ i = 0;
+ while ((name = cpu_model_advertised_features[i++]) != NULL) {
+ ObjectProperty *prop = object_property_find(obj, name, NULL);
+ if (prop) {
+ Error *err = NULL;
+ QObject *value;
+
+ assert(prop->get);
+ value = object_property_get_qobject(obj, name, &err);
+ assert(!err);
+
+ qdict_put_obj(qdict_out, name, value);
+ }
+ }
+
+ if (!qdict_size(qdict_out)) {
+ qobject_unref(qdict_out);
+ } else {
+ expansion_info->model->props = QOBJECT(qdict_out);
+ expansion_info->model->has_props = true;
+ }
+
+ object_unref(obj);
+
+ return expansion_info;
+}
diff --git a/target/arm/translate-vfp.inc.c b/target/arm/translate-vfp.inc.c
index 9ae980b..85c5ef8 100644
--- a/target/arm/translate-vfp.inc.c
+++ b/target/arm/translate-vfp.inc.c
@@ -703,9 +703,10 @@
if (arm_dc_feature(s, ARM_FEATURE_M)) {
/*
* The only M-profile VFP vmrs/vmsr sysreg is FPSCR.
- * Writes to R15 are UNPREDICTABLE; we choose to undef.
+ * Accesses to R15 are UNPREDICTABLE; we choose to undef.
+ * (FPSCR -> r15 is a special case which writes to the PSR flags.)
*/
- if (a->rt == 15 || a->reg != ARM_VFP_FPSCR) {
+ if (a->rt == 15 && (!a->l || a->reg != ARM_VFP_FPSCR)) {
return false;
}
}
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 56f73b4..534ee48 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -262,6 +262,7 @@
check-qtest-sparc64-y += tests/prom-env-test$(EXESUF)
check-qtest-sparc64-y += tests/boot-serial-test$(EXESUF)
+check-qtest-arm-y += tests/arm-cpu-features$(EXESUF)
check-qtest-arm-y += tests/microbit-test$(EXESUF)
check-qtest-arm-y += tests/m25p80-test$(EXESUF)
check-qtest-arm-y += tests/test-arm-mptimer$(EXESUF)
@@ -269,7 +270,8 @@
check-qtest-arm-y += tests/hexloader-test$(EXESUF)
check-qtest-arm-$(CONFIG_PFLASH_CFI02) += tests/pflash-cfi02-test$(EXESUF)
-check-qtest-aarch64-y = tests/numa-test$(EXESUF)
+check-qtest-aarch64-y += tests/arm-cpu-features$(EXESUF)
+check-qtest-aarch64-y += tests/numa-test$(EXESUF)
check-qtest-aarch64-y += tests/boot-serial-test$(EXESUF)
check-qtest-aarch64-y += tests/migration-test$(EXESUF)
# TODO: once aarch64 TCG is fixed on ARM 32 bit host, make test unconditional
@@ -841,6 +843,7 @@
tests/numa-test$(EXESUF): tests/numa-test.o
tests/vmgenid-test$(EXESUF): tests/vmgenid-test.o tests/boot-sector.o tests/acpi-utils.o
tests/cdrom-test$(EXESUF): tests/cdrom-test.o tests/boot-sector.o $(libqos-obj-y)
+tests/arm-cpu-features$(EXESUF): tests/arm-cpu-features.o
tests/migration/stress$(EXESUF): tests/migration/stress.o
$(call quiet-command, $(LINKPROG) -static -O3 $(PTHREAD_LIB) -o $@ $< ,"LINK","$(TARGET_DIR)$@")
diff --git a/tests/arm-cpu-features.c b/tests/arm-cpu-features.c
new file mode 100644
index 0000000..6e99aa9
--- /dev/null
+++ b/tests/arm-cpu-features.c
@@ -0,0 +1,559 @@
+/*
+ * Arm CPU feature test cases
+ *
+ * Copyright (c) 2019 Red Hat Inc.
+ * Authors:
+ * Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qemu/bitops.h"
+#include "libqtest.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qjson.h"
+
+/*
+ * We expect the SVE max-vq to be 16. Also it must be <= 64
+ * for our test code, otherwise 'vls' can't just be a uint64_t.
+ */
+#define SVE_MAX_VQ 16
+
+#define MACHINE "-machine virt,gic-version=max,accel=tcg "
+#define MACHINE_KVM "-machine virt,gic-version=max,accel=kvm:tcg "
+#define QUERY_HEAD "{ 'execute': 'query-cpu-model-expansion', " \
+ " 'arguments': { 'type': 'full', "
+#define QUERY_TAIL "}}"
+
+static bool kvm_enabled(QTestState *qts)
+{
+ QDict *resp, *qdict;
+ bool enabled;
+
+ resp = qtest_qmp(qts, "{ 'execute': 'query-kvm' }");
+ g_assert(qdict_haskey(resp, "return"));
+ qdict = qdict_get_qdict(resp, "return");
+ g_assert(qdict_haskey(qdict, "enabled"));
+ enabled = qdict_get_bool(qdict, "enabled");
+ qobject_unref(resp);
+
+ return enabled;
+}
+
+static QDict *do_query_no_props(QTestState *qts, const char *cpu_type)
+{
+ return qtest_qmp(qts, QUERY_HEAD "'model': { 'name': %s }"
+ QUERY_TAIL, cpu_type);
+}
+
+static QDict *do_query(QTestState *qts, const char *cpu_type,
+ const char *fmt, ...)
+{
+ QDict *resp;
+
+ if (fmt) {
+ QDict *args;
+ va_list ap;
+
+ va_start(ap, fmt);
+ args = qdict_from_vjsonf_nofail(fmt, ap);
+ va_end(ap);
+
+ resp = qtest_qmp(qts, QUERY_HEAD "'model': { 'name': %s, "
+ "'props': %p }"
+ QUERY_TAIL, cpu_type, args);
+ } else {
+ resp = do_query_no_props(qts, cpu_type);
+ }
+
+ return resp;
+}
+
+static const char *resp_get_error(QDict *resp)
+{
+ QDict *qdict;
+
+ g_assert(resp);
+
+ qdict = qdict_get_qdict(resp, "error");
+ if (qdict) {
+ return qdict_get_str(qdict, "desc");
+ }
+
+ return NULL;
+}
+
+#define assert_error(qts, cpu_type, expected_error, fmt, ...) \
+({ \
+ QDict *_resp; \
+ const char *_error; \
+ \
+ _resp = do_query(qts, cpu_type, fmt, ##__VA_ARGS__); \
+ g_assert(_resp); \
+ _error = resp_get_error(_resp); \
+ g_assert(_error); \
+ g_assert(g_str_equal(_error, expected_error)); \
+ qobject_unref(_resp); \
+})
+
+static bool resp_has_props(QDict *resp)
+{
+ QDict *qdict;
+
+ g_assert(resp);
+
+ if (!qdict_haskey(resp, "return")) {
+ return false;
+ }
+ qdict = qdict_get_qdict(resp, "return");
+
+ if (!qdict_haskey(qdict, "model")) {
+ return false;
+ }
+ qdict = qdict_get_qdict(qdict, "model");
+
+ return qdict_haskey(qdict, "props");
+}
+
+static QDict *resp_get_props(QDict *resp)
+{
+ QDict *qdict;
+
+ g_assert(resp);
+ g_assert(resp_has_props(resp));
+
+ qdict = qdict_get_qdict(resp, "return");
+ qdict = qdict_get_qdict(qdict, "model");
+ qdict = qdict_get_qdict(qdict, "props");
+
+ return qdict;
+}
+
+static bool resp_get_feature(QDict *resp, const char *feature)
+{
+ QDict *props;
+
+ g_assert(resp);
+ g_assert(resp_has_props(resp));
+ props = resp_get_props(resp);
+ g_assert(qdict_get(props, feature));
+ return qdict_get_bool(props, feature);
+}
+
+#define assert_has_feature(qts, cpu_type, feature) \
+({ \
+ QDict *_resp = do_query_no_props(qts, cpu_type); \
+ g_assert(_resp); \
+ g_assert(resp_has_props(_resp)); \
+ g_assert(qdict_get(resp_get_props(_resp), feature)); \
+ qobject_unref(_resp); \
+})
+
+#define assert_has_not_feature(qts, cpu_type, feature) \
+({ \
+ QDict *_resp = do_query_no_props(qts, cpu_type); \
+ g_assert(_resp); \
+ g_assert(!resp_has_props(_resp) || \
+ !qdict_get(resp_get_props(_resp), feature)); \
+ qobject_unref(_resp); \
+})
+
+static void assert_type_full(QTestState *qts)
+{
+ const char *error;
+ QDict *resp;
+
+ resp = qtest_qmp(qts, "{ 'execute': 'query-cpu-model-expansion', "
+ "'arguments': { 'type': 'static', "
+ "'model': { 'name': 'foo' }}}");
+ g_assert(resp);
+ error = resp_get_error(resp);
+ g_assert(error);
+ g_assert(g_str_equal(error,
+ "The requested expansion type is not supported"));
+ qobject_unref(resp);
+}
+
+static void assert_bad_props(QTestState *qts, const char *cpu_type)
+{
+ const char *error;
+ QDict *resp;
+
+ resp = qtest_qmp(qts, "{ 'execute': 'query-cpu-model-expansion', "
+ "'arguments': { 'type': 'full', "
+ "'model': { 'name': %s, "
+ "'props': false }}}",
+ cpu_type);
+ g_assert(resp);
+ error = resp_get_error(resp);
+ g_assert(error);
+ g_assert(g_str_equal(error,
+ "Invalid parameter type for 'props', expected: dict"));
+ qobject_unref(resp);
+}
+
+static uint64_t resp_get_sve_vls(QDict *resp)
+{
+ QDict *props;
+ const QDictEntry *e;
+ uint64_t vls = 0;
+ int n = 0;
+
+ g_assert(resp);
+ g_assert(resp_has_props(resp));
+
+ props = resp_get_props(resp);
+
+ for (e = qdict_first(props); e; e = qdict_next(props, e)) {
+ if (strlen(e->key) > 3 && !strncmp(e->key, "sve", 3) &&
+ g_ascii_isdigit(e->key[3])) {
+ char *endptr;
+ int bits;
+
+ bits = g_ascii_strtoll(&e->key[3], &endptr, 10);
+ if (!bits || *endptr != '\0') {
+ continue;
+ }
+
+ if (qdict_get_bool(props, e->key)) {
+ vls |= BIT_ULL((bits / 128) - 1);
+ }
+ ++n;
+ }
+ }
+
+ g_assert(n == SVE_MAX_VQ);
+
+ return vls;
+}
+
+#define assert_sve_vls(qts, cpu_type, expected_vls, fmt, ...) \
+({ \
+ QDict *_resp = do_query(qts, cpu_type, fmt, ##__VA_ARGS__); \
+ g_assert(_resp); \
+ g_assert(resp_has_props(_resp)); \
+ g_assert(resp_get_sve_vls(_resp) == expected_vls); \
+ qobject_unref(_resp); \
+})
+
+static void sve_tests_default(QTestState *qts, const char *cpu_type)
+{
+ /*
+ * With no sve-max-vq or sve<N> properties on the command line
+ * the default is to have all vector lengths enabled. This also
+ * tests that 'sve' is 'on' by default.
+ */
+ assert_sve_vls(qts, cpu_type, BIT_ULL(SVE_MAX_VQ) - 1, NULL);
+
+ /* With SVE off, all vector lengths should also be off. */
+ assert_sve_vls(qts, cpu_type, 0, "{ 'sve': false }");
+
+ /* With SVE on, we must have at least one vector length enabled. */
+ assert_error(qts, cpu_type, "cannot disable sve128", "{ 'sve128': false }");
+
+ /* Basic enable/disable tests. */
+ assert_sve_vls(qts, cpu_type, 0x7, "{ 'sve384': true }");
+ assert_sve_vls(qts, cpu_type, ((BIT_ULL(SVE_MAX_VQ) - 1) & ~BIT_ULL(2)),
+ "{ 'sve384': false }");
+
+ /*
+ * ---------------------------------------------------------------------
+ * power-of-two(vq) all-power- can can
+ * of-two(< vq) enable disable
+ * ---------------------------------------------------------------------
+ * vq < max_vq no MUST* yes yes
+ * vq < max_vq yes MUST* yes no
+ * ---------------------------------------------------------------------
+ * vq == max_vq n/a MUST* yes** yes**
+ * ---------------------------------------------------------------------
+ * vq > max_vq n/a no no yes
+ * vq > max_vq n/a yes yes yes
+ * ---------------------------------------------------------------------
+ *
+ * [*] "MUST" means this requirement must already be satisfied,
+ * otherwise 'max_vq' couldn't itself be enabled.
+ *
+ * [**] Not testable with the QMP interface, only with the command line.
+ */
+
+ /* max_vq := 8 */
+ assert_sve_vls(qts, cpu_type, 0x8b, "{ 'sve1024': true }");
+
+ /* max_vq := 8, vq < max_vq, !power-of-two(vq) */
+ assert_sve_vls(qts, cpu_type, 0x8f,
+ "{ 'sve1024': true, 'sve384': true }");
+ assert_sve_vls(qts, cpu_type, 0x8b,
+ "{ 'sve1024': true, 'sve384': false }");
+
+ /* max_vq := 8, vq < max_vq, power-of-two(vq) */
+ assert_sve_vls(qts, cpu_type, 0x8b,
+ "{ 'sve1024': true, 'sve256': true }");
+ assert_error(qts, cpu_type, "cannot disable sve256",
+ "{ 'sve1024': true, 'sve256': false }");
+
+ /* max_vq := 3, vq > max_vq, !all-power-of-two(< vq) */
+ assert_error(qts, cpu_type, "cannot disable sve512",
+ "{ 'sve384': true, 'sve512': false, 'sve640': true }");
+
+ /*
+ * We can disable power-of-two vector lengths when all larger lengths
+ * are also disabled. We only need to disable the power-of-two length,
+ * as all non-enabled larger lengths will then be auto-disabled.
+ */
+ assert_sve_vls(qts, cpu_type, 0x7, "{ 'sve512': false }");
+
+ /* max_vq := 3, vq > max_vq, all-power-of-two(< vq) */
+ assert_sve_vls(qts, cpu_type, 0x1f,
+ "{ 'sve384': true, 'sve512': true, 'sve640': true }");
+ assert_sve_vls(qts, cpu_type, 0xf,
+ "{ 'sve384': true, 'sve512': true, 'sve640': false }");
+}
+
+static void sve_tests_sve_max_vq_8(const void *data)
+{
+ QTestState *qts;
+
+ qts = qtest_init(MACHINE "-cpu max,sve-max-vq=8");
+
+ assert_sve_vls(qts, "max", BIT_ULL(8) - 1, NULL);
+
+ /*
+ * Disabling the max-vq set by sve-max-vq is not allowed, but
+ * of course enabling it is OK.
+ */
+ assert_error(qts, "max", "cannot disable sve1024", "{ 'sve1024': false }");
+ assert_sve_vls(qts, "max", 0xff, "{ 'sve1024': true }");
+
+ /*
+ * Enabling anything larger than max-vq set by sve-max-vq is not
+ * allowed, but of course disabling everything larger is OK.
+ */
+ assert_error(qts, "max", "cannot enable sve1152", "{ 'sve1152': true }");
+ assert_sve_vls(qts, "max", 0xff, "{ 'sve1152': false }");
+
+ /*
+ * We can enable/disable non power-of-two lengths smaller than the
+ * max-vq set by sve-max-vq, but, while we can enable power-of-two
+ * lengths, we can't disable them.
+ */
+ assert_sve_vls(qts, "max", 0xff, "{ 'sve384': true }");
+ assert_sve_vls(qts, "max", 0xfb, "{ 'sve384': false }");
+ assert_sve_vls(qts, "max", 0xff, "{ 'sve256': true }");
+ assert_error(qts, "max", "cannot disable sve256", "{ 'sve256': false }");
+
+ qtest_quit(qts);
+}
+
+static void sve_tests_sve_off(const void *data)
+{
+ QTestState *qts;
+
+ qts = qtest_init(MACHINE "-cpu max,sve=off");
+
+ /* SVE is off, so the map should be empty. */
+ assert_sve_vls(qts, "max", 0, NULL);
+
+ /* The map stays empty even if we turn lengths off. */
+ assert_sve_vls(qts, "max", 0, "{ 'sve128': false }");
+
+ /* It's an error to enable lengths when SVE is off. */
+ assert_error(qts, "max", "cannot enable sve128", "{ 'sve128': true }");
+
+ /* With SVE re-enabled we should get all vector lengths enabled. */
+ assert_sve_vls(qts, "max", BIT_ULL(SVE_MAX_VQ) - 1, "{ 'sve': true }");
+
+ /* Or enable SVE with just specific vector lengths. */
+ assert_sve_vls(qts, "max", 0x3,
+ "{ 'sve': true, 'sve128': true, 'sve256': true }");
+
+ qtest_quit(qts);
+}
+
+static void sve_tests_sve_off_kvm(const void *data)
+{
+ QTestState *qts;
+
+ qts = qtest_init(MACHINE_KVM "-cpu max,sve=off");
+
+ /*
+ * We don't know if this host supports SVE so we don't
+ * attempt to test enabling anything. We only test that
+ * everything is disabled (as it should be with sve=off)
+ * and that using sve<N>=off to explicitly disable vector
+ * lengths is OK too.
+ */
+ assert_sve_vls(qts, "max", 0, NULL);
+ assert_sve_vls(qts, "max", 0, "{ 'sve128': false }");
+
+ qtest_quit(qts);
+}
+
+static void test_query_cpu_model_expansion(const void *data)
+{
+ QTestState *qts;
+
+ qts = qtest_init(MACHINE "-cpu max");
+
+ /* Test common query-cpu-model-expansion input validation */
+ assert_type_full(qts);
+ assert_bad_props(qts, "max");
+ assert_error(qts, "foo", "The CPU type 'foo' is not a recognized "
+ "ARM CPU type", NULL);
+ assert_error(qts, "max", "Parameter 'not-a-prop' is unexpected",
+ "{ 'not-a-prop': false }");
+ assert_error(qts, "host", "The CPU type 'host' requires KVM", NULL);
+
+ /* Test expected feature presence/absence for some cpu types */
+ assert_has_feature(qts, "max", "pmu");
+ assert_has_feature(qts, "cortex-a15", "pmu");
+ assert_has_not_feature(qts, "cortex-a15", "aarch64");
+
+ if (g_str_equal(qtest_get_arch(), "aarch64")) {
+ assert_has_feature(qts, "max", "aarch64");
+ assert_has_feature(qts, "max", "sve");
+ assert_has_feature(qts, "max", "sve128");
+ assert_has_feature(qts, "cortex-a57", "pmu");
+ assert_has_feature(qts, "cortex-a57", "aarch64");
+
+ sve_tests_default(qts, "max");
+
+ /* Test that features that depend on KVM generate errors without. */
+ assert_error(qts, "max",
+ "'aarch64' feature cannot be disabled "
+ "unless KVM is enabled and 32-bit EL1 "
+ "is supported",
+ "{ 'aarch64': false }");
+ }
+
+ qtest_quit(qts);
+}
+
+static void test_query_cpu_model_expansion_kvm(const void *data)
+{
+ QTestState *qts;
+
+ qts = qtest_init(MACHINE_KVM "-cpu max");
+
+ /*
+ * These tests target the 'host' CPU type, so KVM must be enabled.
+ */
+ if (!kvm_enabled(qts)) {
+ qtest_quit(qts);
+ return;
+ }
+
+ if (g_str_equal(qtest_get_arch(), "aarch64")) {
+ bool kvm_supports_sve;
+ char max_name[8], name[8];
+ uint32_t max_vq, vq;
+ uint64_t vls;
+ QDict *resp;
+ char *error;
+
+ assert_has_feature(qts, "host", "aarch64");
+ assert_has_feature(qts, "host", "pmu");
+
+ assert_error(qts, "cortex-a15",
+ "We cannot guarantee the CPU type 'cortex-a15' works "
+ "with KVM on this host", NULL);
+
+ assert_has_feature(qts, "host", "sve");
+ resp = do_query_no_props(qts, "host");
+ kvm_supports_sve = resp_get_feature(resp, "sve");
+ vls = resp_get_sve_vls(resp);
+ qobject_unref(resp);
+
+ if (kvm_supports_sve) {
+ g_assert(vls != 0);
+ max_vq = 64 - __builtin_clzll(vls);
+ sprintf(max_name, "sve%d", max_vq * 128);
+
+ /* Enabling a supported length is of course fine. */
+ assert_sve_vls(qts, "host", vls, "{ %s: true }", max_name);
+
+ /* Get the next supported length smaller than max-vq. */
+ vq = 64 - __builtin_clzll(vls & ~BIT_ULL(max_vq - 1));
+ if (vq) {
+ /*
+ * We have at least one length smaller than max-vq,
+ * so we can disable max-vq.
+ */
+ assert_sve_vls(qts, "host", (vls & ~BIT_ULL(max_vq - 1)),
+ "{ %s: false }", max_name);
+
+ /*
+ * Smaller, supported vector lengths cannot be disabled
+ * unless all larger, supported vector lengths are also
+ * disabled.
+ */
+ sprintf(name, "sve%d", vq * 128);
+ error = g_strdup_printf("cannot disable %s", name);
+ assert_error(qts, "host", error,
+ "{ %s: true, %s: false }",
+ max_name, name);
+ g_free(error);
+ }
+
+ /*
+ * The smallest, supported vector length is required, because
+ * we need at least one vector length enabled.
+ */
+ vq = __builtin_ffsll(vls);
+ sprintf(name, "sve%d", vq * 128);
+ error = g_strdup_printf("cannot disable %s", name);
+ assert_error(qts, "host", error, "{ %s: false }", name);
+ g_free(error);
+
+ /* Get an unsupported length. */
+ for (vq = 1; vq <= max_vq; ++vq) {
+ if (!(vls & BIT_ULL(vq - 1))) {
+ break;
+ }
+ }
+ if (vq <= SVE_MAX_VQ) {
+ sprintf(name, "sve%d", vq * 128);
+ error = g_strdup_printf("cannot enable %s", name);
+ assert_error(qts, "host", error, "{ %s: true }", name);
+ g_free(error);
+ }
+ } else {
+ g_assert(vls == 0);
+ }
+ } else {
+ assert_has_not_feature(qts, "host", "aarch64");
+ assert_has_not_feature(qts, "host", "pmu");
+ assert_has_not_feature(qts, "host", "sve");
+ }
+
+ qtest_quit(qts);
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ qtest_add_data_func("/arm/query-cpu-model-expansion",
+ NULL, test_query_cpu_model_expansion);
+
+ /*
+ * For now we only run KVM specific tests with AArch64 QEMU in
+ * order avoid attempting to run an AArch32 QEMU with KVM on
+ * AArch64 hosts. That won't work and isn't easy to detect.
+ */
+ if (g_str_equal(qtest_get_arch(), "aarch64")) {
+ qtest_add_data_func("/arm/kvm/query-cpu-model-expansion",
+ NULL, test_query_cpu_model_expansion_kvm);
+ }
+
+ if (g_str_equal(qtest_get_arch(), "aarch64")) {
+ qtest_add_data_func("/arm/max/query-cpu-model-expansion/sve-max-vq-8",
+ NULL, sve_tests_sve_max_vq_8);
+ qtest_add_data_func("/arm/max/query-cpu-model-expansion/sve-off",
+ NULL, sve_tests_sve_off);
+ qtest_add_data_func("/arm/kvm/query-cpu-model-expansion/sve-off",
+ NULL, sve_tests_sve_off_kvm);
+ }
+
+ return g_test_run();
+}
