You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/05/09 18:17:00 UTC
[jira] [Commented] (IMPALA-9690) Bump minimum x86-64 CPU
requirements
[ https://issues.apache.org/jira/browse/IMPALA-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341583#comment-17341583 ]
ASF subversion and git services commented on IMPALA-9690:
---------------------------------------------------------
Commit c10e7c969dfcc4847a8f7708940f4aa1852dbee4 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c10e7c9 ]
IMPALA-9690: Set minimum CPU requirement to AVX2 for x86_64
AVX2 has been available on x86_64 processors for several
years. To simplify the code going forward, this
changes Impala to require AVX2 for x86_64. If AVX2
is not present, Impala will refuse to start up.
Prior to this change, the LLVM IR code for x86_64 was
built with two versions: one that requires SSE4 and
another that does not. This simplifies the code to
have only one version that is always built with AVX2
support.
This does not have any impact on ARM support.
Testing:
- Core job passes
- Release exhaustive job passes
- Performance run on TPC-H scale 42 shows minor improvement.
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(42) | parquet / none / none | 8.38 | -1.02% | 5.28 | -1.01% |
+----------+-----------------------+---------+------------+------------+----------------+
Change-Id: I6de2e897d8ff24dea3c7c1c4ab9d61734839835d
Reviewed-on: http://gerrit.cloudera.org:8080/17406
Reviewed-by: Jim Apple <jb...@apache.org>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Bump minimum x86-64 CPU requirements
> ------------------------------------
>
> Key: IMPALA-9690
> URL: https://issues.apache.org/jira/browse/IMPALA-9690
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: incompatibility, performance
>
> We still have a minimum CPU requirement of SSSE3 support https://impala.apache.org/docs/build/html/topics/impala_prereqs.html. I.e. we don't assume SSE4.2 or AVX or AVX2.
> There is a lot of legacy code to support CPUs without SSE4.2 and various other extensions. As a start, here are all the locations in the code where we branch based on CPU feature:
> {noformat}
> :~/impala/impala$ git grep CpuInfo::IsSupport
> be/src/benchmarks/int-hash-benchmark.cc: if (CpuInfo::IsSupported(CpuInfo::SSE4_2)) suite32.BENCH(uint32_t, CRC);
> be/src/benchmarks/int-hash-benchmark.cc: if (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/benchmarks/int-hash-benchmark.cc: if (CpuInfo::IsSupported(CpuInfo::SSE4_1)) {
> be/src/benchmarks/int-hash-benchmark.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/benchmarks/string-compare-benchmark.cc: if (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.cc: if (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.cc: if (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.inline.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/exec/delimited-text-parser.inline.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
> be/src/runtime/io/disk-io-mgr.cc: if (!CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::SSSE3)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(cpu_info_flag)) {
> be/src/util/bit-util-test.cc:// CpuInfo::IsSupported() checks. This doesn't test the bug precisely but is a canary for
> be/src/util/bit-util.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bit-util.cc: } else if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) {
> be/src/util/bit-util.cc: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) {
> be/src/util/bit-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::POPCNT))) {
> be/src/util/bloom-filter.cc: if (CpuInfo::IsSupported(CpuInfo::AVX)) {
> be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/cpu-info.cc: if (!CpuInfo::IsSupported(CpuInfo::SSSE3)) {
> be/src/util/cpu-info.h: /// // line, CpuInfo::IsSupported(CpuInfo::AVX2) will return false.
> be/src/util/cpu-info.h: : feature_(feature), reenable_(CpuInfo::IsSupported(feature)) {
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
> be/src/util/openssl-util.cc: return (CpuInfo::IsSupported(CpuInfo::PCLMULQDQ)
> {noformat}
> We also ship two versions of the codegen module, one of which (nosse42) is essentially never used.
> I think it would be uncontroversial to bump the minimum requirement to SSE4.2, which would allow us to delete some old fallbacks. I think the last time Intel or AMD shipped a processor without this was 2010 or 2011. Jumping to AVX is probably almost as uncontroversial, since it looks like that has been universal for nearly as long: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX.
> Some older lower-performance cloud instance types don't support AVX, it looks like, but I think this is an edge case.
> It would be very nice to require AVX2 because that could remove a bunch of conditional code (I think the contributors adding ARM support might want to keep the scalar fallback code though, potentially). It looks like most Intel and AMD processors have supported it since 2013 and 2015 respectively, except some low-end Intel processors: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2.
> In the past we've been a bit conservative about this - people do sometimes use very old hardware as a test cluster because it was the hardware that they could get their hands on (e.g. I remember someone complaining several years back that they couldn't get Impala running on some old AMD CPUs that were missing SSSE3 support).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org